honeymoose
/
hapi-fhir
mirror of https://github.com/hapifhir/hapi-fhir.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Reduce storage required for indexing - stop writing sp_name, res_type, and sp_updated to hfj_spidx_* tables (#5941) 

* Reduce storage required for indexing - implementation
This commit is contained in:
volodymyr-korzh 2024-06-20 14:10:40 -06:00 committed by GitHub
parent 5799c6b42b
commit 0397b9ddc8
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
48 changed files with 1837 additions and 266 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

7
hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/7_4_0/5937-reduce-storage-for-sp-index-tables.yaml Normal file
View File

@ -0,0 +1,7 @@
---
type: perf
issue: 5937
title: "A new configuration option, `StorageSettings#setIndexStorageOptimized(boolean)` has been added. If enabled,
the server will not write data to the `SP_NAME`, `RES_TYPE`, `SP_UPDATED` columns for all `HFJ_SPIDX_xxx` tables.
This can help reduce the overall storage size on servers where HFJ_SPIDX tables are expected to have a large
amount of data."

25
hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/7_4_0/upgrade.md
View File

@ -0,0 +1,25 @@
## Possible migration errors on SQL Server (MSSQL)
* This affects only clients running SQL Server (MSSQL) who have custom indexes on `HFJ_SPIDX` tables, which
include `sp_name` or `res_type` columns.
* For those clients, migration of `sp_name` and `res_type` columns to nullable on `HFJ_SPIDX` tables may be completed with errors, as changing a column to nullable when a column is a
part of an index can lead to errors on SQL Server (MSSQL).
* If client wants to use existing indexes and settings, these errors can be ignored. However, if client wants to enable both [Index Storage Optimized](/hapi-fhir/apidocs/hapi-fhir-jpaserver-model/ca/uhn/fhir/jpa/model/entity/StorageSettings.html#setIndexStorageOptimized(boolean))
and [Index Missing Fields](/hapi-fhir/apidocs/hapi-fhir-jpaserver-model/ca/uhn/fhir/jpa/model/entity/StorageSettings.html#getIndexMissingFields()) settings, manual steps are required to change `sp_name` and `res_type` nullability.
To update columns to nullable in such a scenario, execute steps below:
1. Indexes that include `sp_name` or `res_type` columns should be dropped:
```sql
DROP INDEX IDX_SP_TOKEN_REST_TYPE_SP_NAME ON HFJ_SPIDX_TOKEN;
```
2. The nullability of `sp_name` and `res_type` columns should be updated:
```sql
ALTER TABLE HFJ_SPIDX_TOKEN ALTER COLUMN RES_TYPE varchar(100) NULL;
ALTER TABLE HFJ_SPIDX_TOKEN ALTER COLUMN SP_NAME varchar(100) NULL;
```
3. Additionally, the following index may need to be added to improve the search performance:
```sql
CREATE INDEX IDX_SP_TOKEN_MISSING_OPTIMIZED ON HFJ_SPIDX_TOKEN (HASH_IDENTITY, SP_MISSING, RES_ID, PARTITION_ID);
```

16
hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/docs/server_jpa/performance.md
View File

@ -68,3 +68,19 @@ This setting controls whether non-resource (ex: Patient is a resource, MdmLink i
Clients may want to disable this setting for performance reasons as it populates a new set of database tables when enabled.
Setting this property explicitly to false disables the feature: [Non Resource DB History](/apidocs/hapi-fhir-storage/ca/uhn/fhir/jpa/api/config/JpaStorageSettings.html#isNonResourceDbHistoryEnabled())
# Enabling Index Storage Optimization
If enabled, the server will not write data to the `SP_NAME`, `RES_TYPE`, `SP_UPDATED` columns for all `HFJ_SPIDX_xxx` tables.
This setting may be enabled on servers where `HFJ_SPIDX_xxx` tables are expected to have a large amount of data (millions of rows) in order to reduce overall storage size.
Setting this property explicitly to true enables the feature: [Index Storage Optimized](/hapi-fhir/apidocs/hapi-fhir-jpaserver-model/ca/uhn/fhir/jpa/model/entity/StorageSettings.html#setIndexStorageOptimized(boolean))
## Limitations
* This setting only applies to newly inserted and updated rows in `HFJ_SPIDX_xxx` tables. All existing rows will still have values in `SP_NAME`, `RES_TYPE` and `SP_UPDATED` columns. Executing `$reindex` operation will apply storage optimization to existing data.
* If this setting is enabled along with [Index Missing Fields](/hapi-fhir/apidocs/hapi-fhir-jpaserver-model/ca/uhn/fhir/jpa/model/entity/StorageSettings.html#getIndexMissingFields()) setting, the following index may need to be added into the `HFJ_SPIDX_xxx` tables to improve the search performance: `(HASH_IDENTITY, SP_MISSING, RES_ID, PARTITION_ID)`.
* This setting should not be enabled in combination with [Include Partition in Search Hashes](/hapi-fhir/apidocs/hapi-fhir-jpaserver-model/ca/uhn/fhir/jpa/model/config/PartitionSettings.html#setIncludePartitionInSearchHashes(boolean)) flag, as in this case, Partition could not be included in Search Hashes.

4
hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/docs/server_jpa/schema.md
View File

@ -502,7 +502,7 @@ The following columns are common to **all HFJ_SPIDX_xxx tables**.
<td>SP_NAME</td>
<td></td>
<td>String</td>
<td></td>
<td>Nullable</td>
<td>
This is the name of the search parameter being indexed.
</td>
@ -511,7 +511,7 @@ The following columns are common to **all HFJ_SPIDX_xxx tables**.
<td>RES_TYPE</td>
<td></td>
<td>String</td>
<td></td>
<td>Nullable</td>
<td>
This is the name of the resource being indexed.
</td>

2
hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/docs/server_jpa_partitioning/enabling_in_hapi_fhir.md
View File

@ -6,6 +6,6 @@ The [PartitionSettings](/hapi-fhir/apidocs/hapi-fhir-jpaserver-model/ca/uhn/fhir
The following settings can be enabled:
* **Include Partition in Search Hashes** ([JavaDoc](/hapi-fhir/apidocs/hapi-fhir-jpaserver-model/ca/uhn/fhir/jpa/model/config/PartitionSettings.html#setIncludePartitionInSearchHashes(boolean))): If this feature is enabled, partition IDs will be factored into [Search Hashes](/hapi-fhir/docs/server_jpa/schema.html#search-hashes). When this flag is not set (as is the default), when a search requests a specific partition, an additional SQL WHERE predicate is added to the query to explicitly request the given partition ID. When this flag is set, this additional WHERE predicate is not necessary since the partition is factored into the hash value being searched on. Setting this flag avoids the need to manually adjust indexes against the HFJ_SPIDX tables. Note that this flag should **not be used in environments where partitioning is being used for security purposes**, since it is possible for a user to reverse engineer false hash collisions.
* **Include Partition in Search Hashes** ([JavaDoc](/hapi-fhir/apidocs/hapi-fhir-jpaserver-model/ca/uhn/fhir/jpa/model/config/PartitionSettings.html#setIncludePartitionInSearchHashes(boolean))): If this feature is enabled, partition IDs will be factored into [Search Hashes](/hapi-fhir/docs/server_jpa/schema.html#search-hashes). When this flag is not set (as is the default), when a search requests a specific partition, an additional SQL WHERE predicate is added to the query to explicitly request the given partition ID. When this flag is set, this additional WHERE predicate is not necessary since the partition is factored into the hash value being searched on. Setting this flag avoids the need to manually adjust indexes against the HFJ_SPIDX tables. Note that this flag should **not be used in environments where partitioning is being used for security purposes**, since it is possible for a user to reverse engineer false hash collisions. This setting should not be enabled in combination with [Index Storage Optimized](/hapi-fhir/apidocs/hapi-fhir-jpaserver-model/ca/uhn/fhir/jpa/model/entity/StorageSettings.html#isIndexStorageOptimized()) flag, as in this case Partition could not be included in Search Hashes.
* **Cross-Partition Reference Mode**: ([JavaDoc](/hapi-fhir/apidocs/hapi-fhir-jpaserver-model/ca/uhn/fhir/jpa/model/config/PartitionSettings.html#setAllowReferencesAcrossPartitions(ca.uhn.fhir.jpa.model.config.PartitionSettings.CrossPartitionReferenceMode))): This setting controls whether resources in one partition should be allowed to create references to resources in other partitions.

14
hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/config/SearchConfig.java
View File

@ -19,7 +19,9 @@
*/
package ca.uhn.fhir.jpa.config;
import ca.uhn.fhir.context.ConfigurationException;
import ca.uhn.fhir.context.FhirContext;
import ca.uhn.fhir.i18n.Msg;
import ca.uhn.fhir.interceptor.api.IInterceptorBroadcaster;
import ca.uhn.fhir.jpa.api.config.JpaStorageSettings;
import ca.uhn.fhir.jpa.api.dao.DaoRegistry;
@ -47,6 +49,7 @@ import ca.uhn.fhir.jpa.search.cache.ISearchCacheSvc;
import ca.uhn.fhir.jpa.search.cache.ISearchResultCacheSvc;
import ca.uhn.fhir.rest.server.IPagingProvider;
import ca.uhn.fhir.rest.server.util.ISearchParamRegistry;
import jakarta.annotation.PostConstruct;
import org.hl7.fhir.instance.model.api.IBaseResource;
import org.springframework.beans.factory.BeanFactory;
import org.springframework.beans.factory.annotation.Autowired;
@ -206,4 +209,15 @@ public class SearchConfig {
exceptionService() // singleton
);
}
@PostConstruct
public void validateConfiguration() {
if (myStorageSettings.isIndexStorageOptimized()
&& myPartitionSettings.isPartitioningEnabled()
&& myPartitionSettings.isIncludePartitionInSearchHashes()) {
throw new ConfigurationException(Msg.code(2525) + "Incorrect configuration. "
+ "StorageSettings#isIndexStorageOptimized and PartitionSettings.isIncludePartitionInSearchHashes "
+ "cannot be enabled at the same time.");
}
}
}

69
hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/dao/index/DaoSearchParamSynchronizer.java
View File

@ -20,7 +20,9 @@
package ca.uhn.fhir.jpa.dao.index;
import ca.uhn.fhir.jpa.model.entity.BaseResourceIndex;
import ca.uhn.fhir.jpa.model.entity.BaseResourceIndexedSearchParam;
import ca.uhn.fhir.jpa.model.entity.ResourceTable;
import ca.uhn.fhir.jpa.model.entity.StorageSettings;
import ca.uhn.fhir.jpa.searchparam.extractor.ResourceIndexedSearchParams;
import ca.uhn.fhir.jpa.util.AddRemoveCount;
import com.google.common.annotations.VisibleForTesting;
@ -29,10 +31,12 @@ import jakarta.persistence.PersistenceContext;
import jakarta.persistence.PersistenceContextType;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;
import java.util.ArrayList;
import java.util.Collection;
import java.util.Date;
import java.util.HashSet;
import java.util.Iterator;
import java.util.List;
@ -42,6 +46,9 @@ import java.util.Set;
public class DaoSearchParamSynchronizer {
private static final Logger ourLog = LoggerFactory.getLogger(DaoSearchParamSynchronizer.class);
@Autowired
private StorageSettings myStorageSettings;
@PersistenceContext(type = PersistenceContextType.TRANSACTION)
protected EntityManager myEntityManager;
@ -68,6 +75,11 @@ public class DaoSearchParamSynchronizer {
return retVal;
}
@VisibleForTesting
public void setStorageSettings(StorageSettings theStorageSettings) {
this.myStorageSettings = theStorageSettings;
}
@VisibleForTesting
public void setEntityManager(EntityManager theEntityManager) {
myEntityManager = theEntityManager;
@ -115,6 +127,7 @@ public class DaoSearchParamSynchronizer {
List<T> paramsToRemove = subtract(theExistingParams, newParams);
List<T> paramsToAdd = subtract(newParams, theExistingParams);
tryToReuseIndexEntities(paramsToRemove, paramsToAdd);
updateExistingParamsIfRequired(theExistingParams, paramsToAdd, newParams, paramsToRemove);
for (T next : paramsToRemove) {
if (!myEntityManager.contains(next)) {
@ -134,6 +147,62 @@ public class DaoSearchParamSynchronizer {
theAddRemoveCount.addToRemoveCount(paramsToRemove.size());
}
/**
* <p>
* This method performs an update of Search Parameter's fields in the case of
* <code>$reindex</code> or update operation by:
* 1. Marking existing entities for updating to apply index storage optimization,
* if it is enabled (disabled by default).
* 2. Recovering <code>SP_NAME</code>, <code>RES_TYPE</code> values of Search Parameter's fields
* for existing entities in case if index storage optimization is disabled (but was enabled previously).
* </p>
* For details, see: {@link StorageSettings#isIndexStorageOptimized()}
*/
private <T extends BaseResourceIndex> void updateExistingParamsIfRequired(
Collection<T> theExistingParams,
List<T> theParamsToAdd,
Collection<T> theNewParams,
List<T> theParamsToRemove) {
theExistingParams.stream()
.filter(BaseResourceIndexedSearchParam.class::isInstance)
.map(BaseResourceIndexedSearchParam.class::cast)
.filter(this::isSearchParameterUpdateRequired)
.filter(sp -> !theParamsToAdd.contains(sp))
.filter(sp -> !theParamsToRemove.contains(sp))
.forEach(sp -> {
// force hibernate to update Search Parameter entity by resetting SP_UPDATED value
sp.setUpdated(new Date());
recoverExistingSearchParameterIfRequired(sp, theNewParams);
theParamsToAdd.add((T) sp);
});
}
/**
* Search parameters should be updated after changing IndexStorageOptimized setting.
* If IndexStorageOptimized is disabled (and was enabled previously), this method copies paramName
* and Resource Type from extracted to existing search parameter.
*/
private <T extends BaseResourceIndex> void recoverExistingSearchParameterIfRequired(
BaseResourceIndexedSearchParam theSearchParamToRecover, Collection<T> theNewParams) {
if (!myStorageSettings.isIndexStorageOptimized()) {
theNewParams.stream()
.filter(BaseResourceIndexedSearchParam.class::isInstance)
.map(BaseResourceIndexedSearchParam.class::cast)
.filter(paramToAdd -> paramToAdd.equals(theSearchParamToRecover))
.findFirst()
.ifPresent(newParam -> {
theSearchParamToRecover.restoreParamName(newParam.getParamName());
theSearchParamToRecover.setResourceType(newParam.getResourceType());
});
}
}
private boolean isSearchParameterUpdateRequired(BaseResourceIndexedSearchParam theSearchParameter) {
return (myStorageSettings.isIndexStorageOptimized() && !theSearchParameter.isIndexStorageOptimized())
|| (!myStorageSettings.isIndexStorageOptimized() && theSearchParameter.isIndexStorageOptimized());
}
/**
* The logic here is that often times when we update a resource we are dropping
* one index row and adding another. This method tries to reuse rows that would otherwise

98
hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/migrate/tasks/HapiFhirJpaMigrationTasks.java
View File

@ -250,6 +250,104 @@ public class HapiFhirJpaMigrationTasks extends BaseMigrationTasks<VersionEnum> {
.unique(false)
.withColumns("RES_UPDATED", "RES_ID")
.heavyweightSkipByDefault();
// Allow null values in SP_NAME, RES_TYPE columns for all HFJ_SPIDX_* tables. These are marked as failure
// allowed, since SQL Server won't let us change nullability on columns with indexes pointing to them.
{
Builder.BuilderWithTableName spidxCoords = version.onTable("HFJ_SPIDX_COORDS");
spidxCoords
.modifyColumn("20240617.1", "SP_NAME")
.nullable()
.withType(ColumnTypeEnum.STRING, 100)
.failureAllowed();
spidxCoords
.modifyColumn("20240617.2", "RES_TYPE")
.nullable()
.withType(ColumnTypeEnum.STRING, 100)
.failureAllowed();
Builder.BuilderWithTableName spidxDate = version.onTable("HFJ_SPIDX_DATE");
spidxDate
.modifyColumn("20240617.3", "SP_NAME")
.nullable()
.withType(ColumnTypeEnum.STRING, 100)
.failureAllowed();
spidxDate
.modifyColumn("20240617.4", "RES_TYPE")
.nullable()
.withType(ColumnTypeEnum.STRING, 100)
.failureAllowed();
Builder.BuilderWithTableName spidxNumber = version.onTable("HFJ_SPIDX_NUMBER");
spidxNumber
.modifyColumn("20240617.5", "SP_NAME")
.nullable()
.withType(ColumnTypeEnum.STRING, 100)
.failureAllowed();
spidxNumber
.modifyColumn("20240617.6", "RES_TYPE")
.nullable()
.withType(ColumnTypeEnum.STRING, 100)
.failureAllowed();
Builder.BuilderWithTableName spidxQuantity = version.onTable("HFJ_SPIDX_QUANTITY");
spidxQuantity
.modifyColumn("20240617.7", "SP_NAME")
.nullable()
.withType(ColumnTypeEnum.STRING, 100)
.failureAllowed();
spidxQuantity
.modifyColumn("20240617.8", "RES_TYPE")
.nullable()
.withType(ColumnTypeEnum.STRING, 100)
.failureAllowed();
Builder.BuilderWithTableName spidxQuantityNorm = version.onTable("HFJ_SPIDX_QUANTITY_NRML");
spidxQuantityNorm
.modifyColumn("20240617.9", "SP_NAME")
.nullable()
.withType(ColumnTypeEnum.STRING, 100)
.failureAllowed();
spidxQuantityNorm
.modifyColumn("20240617.10", "RES_TYPE")
.nullable()
.withType(ColumnTypeEnum.STRING, 100)
.failureAllowed();
Builder.BuilderWithTableName spidxString = version.onTable("HFJ_SPIDX_STRING");
spidxString
.modifyColumn("20240617.11", "SP_NAME")
.nullable()
.withType(ColumnTypeEnum.STRING, 100)
.failureAllowed();
spidxString
.modifyColumn("20240617.12", "RES_TYPE")
.nullable()
.withType(ColumnTypeEnum.STRING, 100)
.failureAllowed();
Builder.BuilderWithTableName spidxToken = version.onTable("HFJ_SPIDX_TOKEN");
spidxToken
.modifyColumn("20240617.13", "SP_NAME")
.nullable()
.withType(ColumnTypeEnum.STRING, 100)
.failureAllowed();
spidxToken
.modifyColumn("20240617.14", "RES_TYPE")
.nullable()
.withType(ColumnTypeEnum.STRING, 100)
.failureAllowed();
Builder.BuilderWithTableName spidxUri = version.onTable("HFJ_SPIDX_URI");
spidxUri.modifyColumn("20240617.15", "SP_NAME")
.nullable()
.withType(ColumnTypeEnum.STRING, 100)
.failureAllowed();
spidxUri.modifyColumn("20240617.16", "RES_TYPE")
.nullable()
.withType(ColumnTypeEnum.STRING, 100)
.failureAllowed();
}
}
protected void init720() {

17
hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/search/builder/predicate/BaseSearchParamPredicateBuilder.java
View File

@ -98,10 +98,19 @@ public abstract class BaseSearchParamPredicateBuilder extends BaseJoiningPredica
public Condition createPredicateParamMissingForNonReference(
String theResourceName, String theParamName, Boolean theMissing, RequestPartitionId theRequestPartitionId) {
ComboCondition condition = ComboCondition.and(
BinaryCondition.equalTo(getResourceTypeColumn(), generatePlaceholder(theResourceName)),
BinaryCondition.equalTo(getColumnParamName(), generatePlaceholder(theParamName)),
BinaryCondition.equalTo(getMissingColumn(), generatePlaceholder(theMissing)));
List<Condition> conditions = new ArrayList<>();
if (getStorageSettings().isIndexStorageOptimized()) {
Long hashIdentity = BaseResourceIndexedSearchParam.calculateHashIdentity(
getPartitionSettings(), getRequestPartitionId(), theResourceName, theParamName);
conditions.add(BinaryCondition.equalTo(getColumnHashIdentity(), generatePlaceholder(hashIdentity)));
} else {
conditions.add(BinaryCondition.equalTo(getResourceTypeColumn(), generatePlaceholder(theResourceName)));
conditions.add(BinaryCondition.equalTo(getColumnParamName(), generatePlaceholder(theParamName)));
}
conditions.add(BinaryCondition.equalTo(getMissingColumn(), generatePlaceholder(theMissing)));
ComboCondition condition = ComboCondition.and(conditions.toArray());
return combineWithRequestPartitionIdPredicate(theRequestPartitionId, condition);
}

2
hapi-fhir-jpaserver-base/src/test/java/ca/uhn/fhir/jpa/dao/index/DaoSearchParamSynchronizerTest.java
View File

@ -4,6 +4,7 @@ import ca.uhn.fhir.jpa.model.config.PartitionSettings;
import ca.uhn.fhir.jpa.model.entity.BaseResourceIndex;
import ca.uhn.fhir.jpa.model.entity.ResourceIndexedSearchParamNumber;
import ca.uhn.fhir.jpa.model.entity.ResourceTable;
import ca.uhn.fhir.jpa.model.entity.StorageSettings;
import ca.uhn.fhir.jpa.searchparam.extractor.ResourceIndexedSearchParams;
import ca.uhn.fhir.jpa.util.AddRemoveCount;
import jakarta.persistence.EntityManager;
@ -61,6 +62,7 @@ public class DaoSearchParamSynchronizerTest {
THE_SEARCH_PARAM_NUMBER.setResource(resourceTable);
subject.setEntityManager(entityManager);
subject.setStorageSettings(new StorageSettings());
}
@Test

8
hapi-fhir-jpaserver-model/src/main/java/ca/uhn/fhir/jpa/model/config/PartitionSettings.java
View File

@ -19,6 +19,8 @@
*/
package ca.uhn.fhir.jpa.model.config;
import ca.uhn.fhir.jpa.model.entity.StorageSettings;
/**
* @since 5.0.0
*/
@ -58,6 +60,9 @@ public class PartitionSettings {
* <p>
* This setting has no effect if partitioning is not enabled via {@link #isPartitioningEnabled()}.
* </p>
* <p>
* If {@link StorageSettings#isIndexStorageOptimized()} is enabled this setting should be set to <code>false</code>.
* </p>
*/
public boolean isIncludePartitionInSearchHashes() {
return myIncludePartitionInSearchHashes;
@ -71,6 +76,9 @@ public class PartitionSettings {
* <p>
* This setting has no effect if partitioning is not enabled via {@link #isPartitioningEnabled()}.
* </p>
* <p>
* If {@link StorageSettings#isIndexStorageOptimized()} is enabled this setting should be set to <code>false</code>.
* </p>
*/
public PartitionSettings setIncludePartitionInSearchHashes(boolean theIncludePartitionInSearchHashes) {
myIncludePartitionInSearchHashes = theIncludePartitionInSearchHashes;

104
hapi-fhir-jpaserver-model/src/main/java/ca/uhn/fhir/jpa/model/entity/BaseResourceIndexedSearchParam.java
View File

@ -22,15 +22,9 @@ package ca.uhn.fhir.jpa.model.entity;
import ca.uhn.fhir.i18n.Msg;
import ca.uhn.fhir.interceptor.model.RequestPartitionId;
import ca.uhn.fhir.jpa.model.config.PartitionSettings;
import ca.uhn.fhir.jpa.model.util.SearchParamHash;
import ca.uhn.fhir.model.api.IQueryParameterType;
import ca.uhn.fhir.rest.api.Constants;
import ca.uhn.fhir.rest.server.exceptions.InternalErrorException;
import ca.uhn.fhir.util.UrlUtil;
import com.google.common.base.Charsets;
import com.google.common.hash.HashCode;
import com.google.common.hash.HashFunction;
import com.google.common.hash.Hasher;
import com.google.common.hash.Hashing;
import jakarta.persistence.Column;
import jakarta.persistence.MappedSuperclass;
import jakarta.persistence.Temporal;
@ -46,16 +40,6 @@ import java.util.List;
@MappedSuperclass
public abstract class BaseResourceIndexedSearchParam extends BaseResourceIndex {
static final int MAX_SP_NAME = 100;
/**
* Don't change this without careful consideration. You will break existing hashes!
*/
private static final HashFunction HASH_FUNCTION = Hashing.murmur3_128(0);
/**
* Don't make this public 'cause nobody better be able to modify it!
*/
private static final byte[] DELIMITER_BYTES = "|".getBytes(Charsets.UTF_8);
private static final long serialVersionUID = 1L;
@GenericField
@ -63,18 +47,26 @@ public abstract class BaseResourceIndexedSearchParam extends BaseResourceIndex {
private boolean myMissing = false;
@FullTextField
@Column(name = "SP_NAME", length = MAX_SP_NAME, nullable = false)
@Column(name = "SP_NAME", length = MAX_SP_NAME)
private String myParamName;
@Column(name = "RES_ID", insertable = false, updatable = false, nullable = false)
private Long myResourcePid;
@FullTextField
@Column(name = "RES_TYPE", updatable = false, nullable = false, length = Constants.MAX_RESOURCE_NAME_LENGTH)
@Column(name = "RES_TYPE", length = Constants.MAX_RESOURCE_NAME_LENGTH)
private String myResourceType;
/**
* Composite of resourceType, paramName, and partition info if configured.
* Combined with the various date fields for a query.
* Nullable to allow optimized storage.
*/
@Column(name = "HASH_IDENTITY", nullable = true)
protected Long myHashIdentity;
@GenericField
@Column(name = "SP_UPDATED", nullable = true) // TODO: make this false after HAPI 2.3
@Column(name = "SP_UPDATED")
@Temporal(TemporalType.TIMESTAMP)
private Date myUpdated;
@ -98,6 +90,28 @@ public abstract class BaseResourceIndexedSearchParam extends BaseResourceIndex {
}
}
/**
* Restore SP_NAME without clearing hashes
*/
public void restoreParamName(String theParamName) {
if (myParamName == null) {
myParamName = theParamName;
}
}
/**
* Set SP_NAME, RES_TYPE, SP_UPDATED to null without clearing hashes
*/
public void optimizeIndexStorage() {
myParamName = null;
myResourceType = null;
myUpdated = null;
}
public boolean isIndexStorageOptimized() {
return myParamName == null || myResourceType == null || myUpdated == null;
}
// MB pushed these down to the individual SP classes so we could name the FK in the join annotation
/**
* Get the Resource this SP indexes
@ -111,6 +125,7 @@ public abstract class BaseResourceIndexedSearchParam extends BaseResourceIndex {
BaseResourceIndexedSearchParam source = (BaseResourceIndexedSearchParam) theSource;
myMissing = source.myMissing;
myParamName = source.myParamName;
myResourceType = source.myResourceType;
myUpdated = source.myUpdated;
myStorageSettings = source.myStorageSettings;
myPartitionSettings = source.myPartitionSettings;
@ -129,6 +144,14 @@ public abstract class BaseResourceIndexedSearchParam extends BaseResourceIndex {
myResourceType = theResourceType;
}
public void setHashIdentity(Long theHashIdentity) {
myHashIdentity = theHashIdentity;
}
public Long getHashIdentity() {
return myHashIdentity;
}
public Date getUpdated() {
return myUpdated;
}
@ -184,7 +207,8 @@ public abstract class BaseResourceIndexedSearchParam extends BaseResourceIndex {
RequestPartitionId theRequestPartitionId,
String theResourceType,
String theParamName) {
return hash(thePartitionSettings, theRequestPartitionId, theResourceType, theParamName);
return SearchParamHash.hashSearchParam(
thePartitionSettings, theRequestPartitionId, theResourceType, theParamName);
}
public static long calculateHashIdentity(
@ -200,42 +224,6 @@ public abstract class BaseResourceIndexedSearchParam extends BaseResourceIndex {
values[i + 2] = theAdditionalValues.get(i);
}
return hash(thePartitionSettings, theRequestPartitionId, values);
}
/**
* Applies a fast and consistent hashing algorithm to a set of strings
*/
static long hash(
PartitionSettings thePartitionSettings, RequestPartitionId theRequestPartitionId, String... theValues) {
Hasher hasher = HASH_FUNCTION.newHasher();
if (thePartitionSettings.isPartitioningEnabled()
&& thePartitionSettings.isIncludePartitionInSearchHashes()
&& theRequestPartitionId != null) {
if (theRequestPartitionId.getPartitionIds().size() > 1) {
throw new InternalErrorException(Msg.code(1527)
+ "Can not search multiple partitions when partitions are included in search hashes");
}
Integer partitionId = theRequestPartitionId.getFirstPartitionIdOrNull();
if (partitionId != null) {
hasher.putInt(partitionId);
}
}
for (String next : theValues) {
if (next == null) {
hasher.putByte((byte) 0);
} else {
next = UrlUtil.escapeUrlParam(next);
byte[] bytes = next.getBytes(Charsets.UTF_8);
hasher.putBytes(bytes);
}
hasher.putBytes(DELIMITER_BYTES);
}
HashCode hashCode = hasher.hash();
long retVal = hashCode.asLong();
return retVal;
return SearchParamHash.hashSearchParam(thePartitionSettings, theRequestPartitionId, values);
}
}

22
hapi-fhir-jpaserver-model/src/main/java/ca/uhn/fhir/jpa/model/entity/BaseResourceIndexedSearchParamQuantity.java
View File

@ -26,6 +26,8 @@ import jakarta.persistence.MappedSuperclass;
import org.apache.commons.lang3.builder.HashCodeBuilder;
import org.hibernate.search.mapper.pojo.mapping.definition.annotation.FullTextField;
import static ca.uhn.fhir.jpa.model.util.SearchParamHash.hashSearchParam;
@MappedSuperclass
public abstract class BaseResourceIndexedSearchParamQuantity extends BaseResourceIndexedSearchParam {
@ -51,11 +53,6 @@ public abstract class BaseResourceIndexedSearchParamQuantity extends BaseResourc
*/
@Column(name = "HASH_IDENTITY_SYS_UNITS", nullable = true)
private Long myHashIdentitySystemAndUnits;
/**
* @since 3.5.0 - At some point this should be made not-null
*/
@Column(name = "HASH_IDENTITY", nullable = true)
private Long myHashIdentity;
/**
* Constructor
@ -88,14 +85,6 @@ public abstract class BaseResourceIndexedSearchParamQuantity extends BaseResourc
getPartitionSettings(), getPartitionId(), resourceType, paramName, system, units));
}
public Long getHashIdentity() {
return myHashIdentity;
}
public void setHashIdentity(Long theHashIdentity) {
myHashIdentity = theHashIdentity;
}
public Long getHashIdentityAndUnits() {
return myHashIdentityAndUnits;
}
@ -131,8 +120,6 @@ public abstract class BaseResourceIndexedSearchParamQuantity extends BaseResourc
@Override
public int hashCode() {
HashCodeBuilder b = new HashCodeBuilder();
b.append(getResourceType());
b.append(getParamName());
b.append(getHashIdentity());
b.append(getHashIdentityAndUnits());
b.append(getHashIdentitySystemAndUnits());
@ -158,7 +145,8 @@ public abstract class BaseResourceIndexedSearchParamQuantity extends BaseResourc
String theParamName,
String theSystem,
String theUnits) {
return hash(thePartitionSettings, theRequestPartitionId, theResourceType, theParamName, theSystem, theUnits);
return hashSearchParam(
thePartitionSettings, theRequestPartitionId, theResourceType, theParamName, theSystem, theUnits);
}
public static long calculateHashUnits(
@ -177,6 +165,6 @@ public abstract class BaseResourceIndexedSearchParamQuantity extends BaseResourc
String theResourceType,
String theParamName,
String theUnits) {
return hash(thePartitionSettings, theRequestPartitionId, theResourceType, theParamName, theUnits);
return hashSearchParam(thePartitionSettings, theRequestPartitionId, theResourceType, theParamName, theUnits);
}
}

6
hapi-fhir-jpaserver-model/src/main/java/ca/uhn/fhir/jpa/model/entity/ResourceIndexedComboTokenNonUnique.java
View File

@ -39,7 +39,7 @@ import org.apache.commons.lang3.builder.HashCodeBuilder;
import org.apache.commons.lang3.builder.ToStringBuilder;
import org.hl7.fhir.instance.model.api.IIdType;
import static ca.uhn.fhir.jpa.model.entity.BaseResourceIndexedSearchParam.hash;
import static ca.uhn.fhir.jpa.model.util.SearchParamHash.hashSearchParam;
@Entity
@Table(
@ -206,12 +206,12 @@ public class ResourceIndexedComboTokenNonUnique extends BaseResourceIndex
public static long calculateHashComplete(
PartitionSettings partitionSettings, PartitionablePartitionId thePartitionId, String queryString) {
RequestPartitionId requestPartitionId = PartitionablePartitionId.toRequestPartitionId(thePartitionId);
return hash(partitionSettings, requestPartitionId, queryString);
return hashSearchParam(partitionSettings, requestPartitionId, queryString);
}
public static long calculateHashComplete(
PartitionSettings partitionSettings, RequestPartitionId partitionId, String queryString) {
return hash(partitionSettings, partitionId, queryString);
return hashSearchParam(partitionSettings, partitionId, queryString);
}
/**

19
hapi-fhir-jpaserver-model/src/main/java/ca/uhn/fhir/jpa/model/entity/ResourceIndexedSearchParamCoords.java
View File

@ -20,11 +20,13 @@
package ca.uhn.fhir.jpa.model.entity;
import ca.uhn.fhir.jpa.model.config.PartitionSettings;
import ca.uhn.fhir.jpa.model.listener.IndexStorageOptimizationListener;
import ca.uhn.fhir.model.api.IQueryParameterType;
import jakarta.annotation.Nullable;
import jakarta.persistence.Column;
import jakarta.persistence.Embeddable;
import jakarta.persistence.Entity;
import jakarta.persistence.EntityListeners;
import jakarta.persistence.FetchType;
import jakarta.persistence.ForeignKey;
import jakarta.persistence.GeneratedValue;
@ -41,6 +43,7 @@ import org.apache.commons.lang3.builder.ToStringBuilder;
import org.apache.commons.lang3.builder.ToStringStyle;
@Embeddable
@EntityListeners(IndexStorageOptimizationListener.class)
@Entity
@Table(
name = "HFJ_SPIDX_COORDS",
@ -68,11 +71,6 @@ public class ResourceIndexedSearchParamCoords extends BaseResourceIndexedSearchP
@GeneratedValue(strategy = GenerationType.AUTO, generator = "SEQ_SPIDX_COORDS")
@Column(name = "SP_ID")
private Long myId;
/**
* @since 3.5.0 - At some point this should be made not-null
*/
@Column(name = "HASH_IDENTITY", nullable = true)
private Long myHashIdentity;
@ManyToOne(
optional = false,
@ -130,8 +128,7 @@ public class ResourceIndexedSearchParamCoords extends BaseResourceIndexedSearchP
}
ResourceIndexedSearchParamCoords obj = (ResourceIndexedSearchParamCoords) theObj;
EqualsBuilder b = new EqualsBuilder();
b.append(getResourceType(), obj.getResourceType());
b.append(getParamName(), obj.getParamName());
b.append(getHashIdentity(), obj.getHashIdentity());
b.append(getLatitude(), obj.getLatitude());
b.append(getLongitude(), obj.getLongitude());
b.append(isMissing(), obj.isMissing());
@ -147,10 +144,6 @@ public class ResourceIndexedSearchParamCoords extends BaseResourceIndexedSearchP
myHashIdentity = source.myHashIdentity;
}
public void setHashIdentity(Long theHashIdentity) {
myHashIdentity = theHashIdentity;
}
@Override
public Long getId() {
return myId;
@ -184,10 +177,10 @@ public class ResourceIndexedSearchParamCoords extends BaseResourceIndexedSearchP
@Override
public int hashCode() {
HashCodeBuilder b = new HashCodeBuilder();
b.append(getParamName());
b.append(getResourceType());
b.append(getHashIdentity());
b.append(getLatitude());
b.append(getLongitude());
b.append(isMissing());
return b.toHashCode();
}

24
hapi-fhir-jpaserver-model/src/main/java/ca/uhn/fhir/jpa/model/entity/ResourceIndexedSearchParamDate.java
View File

@ -20,6 +20,7 @@
package ca.uhn.fhir.jpa.model.entity;
import ca.uhn.fhir.jpa.model.config.PartitionSettings;
import ca.uhn.fhir.jpa.model.listener.IndexStorageOptimizationListener;
import ca.uhn.fhir.model.api.IQueryParameterType;
import ca.uhn.fhir.model.api.TemporalPrecisionEnum;
import ca.uhn.fhir.model.primitive.InstantDt;
@ -29,6 +30,7 @@ import ca.uhn.fhir.util.DateUtils;
import jakarta.persistence.Column;
import jakarta.persistence.Embeddable;
import jakarta.persistence.Entity;
import jakarta.persistence.EntityListeners;
import jakarta.persistence.FetchType;
import jakarta.persistence.ForeignKey;
import jakarta.persistence.GeneratedValue;
@ -55,6 +57,7 @@ import java.text.SimpleDateFormat;
import java.util.Date;
@Embeddable
@EntityListeners(IndexStorageOptimizationListener.class)
@Entity
@Table(
name = "HFJ_SPIDX_DATE",
@ -109,14 +112,6 @@ public class ResourceIndexedSearchParamDate extends BaseResourceIndexedSearchPar
@Column(name = "SP_ID")
private Long myId;
/**
* Composite of resourceType, paramName, and partition info if configured.
* Combined with the various date fields for a query.
* @since 3.5.0 - At some point this should be made not-null
*/
@Column(name = "HASH_IDENTITY", nullable = true)
private Long myHashIdentity;
@ManyToOne(
optional = false,
fetch = FetchType.LAZY,
@ -264,8 +259,7 @@ public class ResourceIndexedSearchParamDate extends BaseResourceIndexedSearchPar
}
ResourceIndexedSearchParamDate obj = (ResourceIndexedSearchParamDate) theObj;
EqualsBuilder b = new EqualsBuilder();
b.append(getResourceType(), obj.getResourceType());
b.append(getParamName(), obj.getParamName());
b.append(getHashIdentity(), obj.getHashIdentity());
b.append(getTimeFromDate(getValueHigh()), getTimeFromDate(obj.getValueHigh()));
b.append(getTimeFromDate(getValueLow()), getTimeFromDate(obj.getValueLow()));
b.append(getValueLowDateOrdinal(), obj.getValueLowDateOrdinal());
@ -274,10 +268,6 @@ public class ResourceIndexedSearchParamDate extends BaseResourceIndexedSearchPar
return b.isEquals();
}
public void setHashIdentity(Long theHashIdentity) {
myHashIdentity = theHashIdentity;
}
@Override
public Long getId() {
return myId;
@ -316,10 +306,12 @@ public class ResourceIndexedSearchParamDate extends BaseResourceIndexedSearchPar
@Override
public int hashCode() {
HashCodeBuilder b = new HashCodeBuilder();
b.append(getResourceType());
b.append(getParamName());
b.append(getHashIdentity());
b.append(getTimeFromDate(getValueHigh()));
b.append(getTimeFromDate(getValueLow()));
b.append(getValueHighDateOrdinal());
b.append(getValueLowDateOrdinal());
b.append(isMissing());
return b.toHashCode();
}

20
hapi-fhir-jpaserver-model/src/main/java/ca/uhn/fhir/jpa/model/entity/ResourceIndexedSearchParamNumber.java
View File

@ -20,11 +20,13 @@
package ca.uhn.fhir.jpa.model.entity;
import ca.uhn.fhir.jpa.model.config.PartitionSettings;
import ca.uhn.fhir.jpa.model.listener.IndexStorageOptimizationListener;
import ca.uhn.fhir.model.api.IQueryParameterType;
import ca.uhn.fhir.rest.param.NumberParam;
import jakarta.persistence.Column;
import jakarta.persistence.Embeddable;
import jakarta.persistence.Entity;
import jakarta.persistence.EntityListeners;
import jakarta.persistence.FetchType;
import jakarta.persistence.ForeignKey;
import jakarta.persistence.GeneratedValue;
@ -47,6 +49,7 @@ import java.math.BigDecimal;
import java.util.Objects;
@Embeddable
@EntityListeners(IndexStorageOptimizationListener.class)
@Entity
@Table(
name = "HFJ_SPIDX_NUMBER",
@ -69,11 +72,6 @@ public class ResourceIndexedSearchParamNumber extends BaseResourceIndexedSearchP
@GeneratedValue(strategy = GenerationType.AUTO, generator = "SEQ_SPIDX_NUMBER")
@Column(name = "SP_ID")
private Long myId;
/**
* @since 3.5.0 - At some point this should be made not-null
*/
@Column(name = "HASH_IDENTITY", nullable = true)
private Long myHashIdentity;
@ManyToOne(
optional = false,
@ -120,10 +118,6 @@ public class ResourceIndexedSearchParamNumber extends BaseResourceIndexedSearchP
setHashIdentity(calculateHashIdentity(getPartitionSettings(), getPartitionId(), resourceType, paramName));
}
public Long getHashIdentity() {
return myHashIdentity;
}
@Override
public boolean equals(Object theObj) {
if (this == theObj) {
@ -137,8 +131,6 @@ public class ResourceIndexedSearchParamNumber extends BaseResourceIndexedSearchP
}
ResourceIndexedSearchParamNumber obj = (ResourceIndexedSearchParamNumber) theObj;
EqualsBuilder b = new EqualsBuilder();
b.append(getResourceType(), obj.getResourceType());
b.append(getParamName(), obj.getParamName());
b.append(getHashIdentity(), obj.getHashIdentity());
b.append(normalizeForEqualityComparison(getValue()), normalizeForEqualityComparison(obj.getValue()));
b.append(isMissing(), obj.isMissing());
@ -152,10 +144,6 @@ public class ResourceIndexedSearchParamNumber extends BaseResourceIndexedSearchP
return theValue.doubleValue();
}
public void setHashIdentity(Long theHashIdentity) {
myHashIdentity = theHashIdentity;
}
@Override
public Long getId() {
return myId;
@ -177,8 +165,6 @@ public class ResourceIndexedSearchParamNumber extends BaseResourceIndexedSearchP
@Override
public int hashCode() {
HashCodeBuilder b = new HashCodeBuilder();
b.append(getResourceType());
b.append(getParamName());
b.append(getHashIdentity());
b.append(normalizeForEqualityComparison(getValue()));
b.append(isMissing());

17
hapi-fhir-jpaserver-model/src/main/java/ca/uhn/fhir/jpa/model/entity/ResourceIndexedSearchParamQuantity.java
View File

@ -20,11 +20,13 @@
package ca.uhn.fhir.jpa.model.entity;
import ca.uhn.fhir.jpa.model.config.PartitionSettings;
import ca.uhn.fhir.jpa.model.listener.IndexStorageOptimizationListener;
import ca.uhn.fhir.model.api.IQueryParameterType;
import ca.uhn.fhir.rest.param.QuantityParam;
import jakarta.persistence.Column;
import jakarta.persistence.Embeddable;
import jakarta.persistence.Entity;
import jakarta.persistence.EntityListeners;
import jakarta.persistence.FetchType;
import jakarta.persistence.ForeignKey;
import jakarta.persistence.GeneratedValue;
@ -36,6 +38,7 @@ import jakarta.persistence.ManyToOne;
import jakarta.persistence.SequenceGenerator;
import jakarta.persistence.Table;
import org.apache.commons.lang3.builder.EqualsBuilder;
import org.apache.commons.lang3.builder.HashCodeBuilder;
import org.apache.commons.lang3.builder.ToStringBuilder;
import org.apache.commons.lang3.builder.ToStringStyle;
import org.hibernate.search.mapper.pojo.mapping.definition.annotation.ScaledNumberField;
@ -48,6 +51,7 @@ import static org.apache.commons.lang3.StringUtils.isBlank;
// @formatter:off
@Embeddable
@EntityListeners(IndexStorageOptimizationListener.class)
@Entity
@Table(
name = "HFJ_SPIDX_QUANTITY",
@ -173,8 +177,6 @@ public class ResourceIndexedSearchParamQuantity extends BaseResourceIndexedSearc
}
ResourceIndexedSearchParamQuantity obj = (ResourceIndexedSearchParamQuantity) theObj;
EqualsBuilder b = new EqualsBuilder();
b.append(getResourceType(), obj.getResourceType());
b.append(getParamName(), obj.getParamName());
b.append(getHashIdentity(), obj.getHashIdentity());
b.append(getHashIdentityAndUnits(), obj.getHashIdentityAndUnits());
b.append(getHashIdentitySystemAndUnits(), obj.getHashIdentitySystemAndUnits());
@ -183,6 +185,17 @@ public class ResourceIndexedSearchParamQuantity extends BaseResourceIndexedSearc
return b.isEquals();
}
@Override
public int hashCode() {
HashCodeBuilder b = new HashCodeBuilder();
b.append(getHashIdentity());
b.append(getHashIdentityAndUnits());
b.append(getHashIdentitySystemAndUnits());
b.append(isMissing());
b.append(getValue());
return b.toHashCode();
}
@Override
public boolean matches(IQueryParameterType theParam) {

17
hapi-fhir-jpaserver-model/src/main/java/ca/uhn/fhir/jpa/model/entity/ResourceIndexedSearchParamQuantityNormalized.java
View File

@ -20,12 +20,14 @@
package ca.uhn.fhir.jpa.model.entity;
import ca.uhn.fhir.jpa.model.config.PartitionSettings;
import ca.uhn.fhir.jpa.model.listener.IndexStorageOptimizationListener;
import ca.uhn.fhir.jpa.model.util.UcumServiceUtil;
import ca.uhn.fhir.model.api.IQueryParameterType;
import ca.uhn.fhir.rest.param.QuantityParam;
import jakarta.persistence.Column;
import jakarta.persistence.Embeddable;
import jakarta.persistence.Entity;
import jakarta.persistence.EntityListeners;
import jakarta.persistence.FetchType;
import jakarta.persistence.ForeignKey;
import jakarta.persistence.GeneratedValue;
@ -37,6 +39,7 @@ import jakarta.persistence.ManyToOne;
import jakarta.persistence.SequenceGenerator;
import jakarta.persistence.Table;
import org.apache.commons.lang3.builder.EqualsBuilder;
import org.apache.commons.lang3.builder.HashCodeBuilder;
import org.apache.commons.lang3.builder.ToStringBuilder;
import org.apache.commons.lang3.builder.ToStringStyle;
import org.fhir.ucum.Pair;
@ -50,6 +53,7 @@ import static org.apache.commons.lang3.StringUtils.isBlank;
// @formatter:off
@Embeddable
@EntityListeners(IndexStorageOptimizationListener.class)
@Entity
@Table(
name = "HFJ_SPIDX_QUANTITY_NRML",
@ -189,8 +193,6 @@ public class ResourceIndexedSearchParamQuantityNormalized extends BaseResourceIn
}
ResourceIndexedSearchParamQuantityNormalized obj = (ResourceIndexedSearchParamQuantityNormalized) theObj;
EqualsBuilder b = new EqualsBuilder();
b.append(getResourceType(), obj.getResourceType());
b.append(getParamName(), obj.getParamName());
b.append(getHashIdentity(), obj.getHashIdentity());
b.append(getHashIdentityAndUnits(), obj.getHashIdentityAndUnits());
b.append(getHashIdentitySystemAndUnits(), obj.getHashIdentitySystemAndUnits());
@ -199,6 +201,17 @@ public class ResourceIndexedSearchParamQuantityNormalized extends BaseResourceIn
return b.isEquals();
}
@Override
public int hashCode() {
HashCodeBuilder b = new HashCodeBuilder();
b.append(getHashIdentity());
b.append(getHashIdentityAndUnits());
b.append(getHashIdentitySystemAndUnits());
b.append(isMissing());
b.append(getValue());
return b.toHashCode();
}
@Override
public boolean matches(IQueryParameterType theParam) {

28
hapi-fhir-jpaserver-model/src/main/java/ca/uhn/fhir/jpa/model/entity/ResourceIndexedSearchParamString.java
View File

@ -22,12 +22,14 @@ package ca.uhn.fhir.jpa.model.entity;
import ca.uhn.fhir.i18n.Msg;
import ca.uhn.fhir.interceptor.model.RequestPartitionId;
import ca.uhn.fhir.jpa.model.config.PartitionSettings;
import ca.uhn.fhir.jpa.model.listener.IndexStorageOptimizationListener;
import ca.uhn.fhir.model.api.IQueryParameterType;
import ca.uhn.fhir.rest.param.StringParam;
import ca.uhn.fhir.util.StringUtil;
import jakarta.persistence.Column;
import jakarta.persistence.Embeddable;
import jakarta.persistence.Entity;
import jakarta.persistence.EntityListeners;
import jakarta.persistence.ForeignKey;
import jakarta.persistence.GeneratedValue;
import jakarta.persistence.GenerationType;
@ -42,10 +44,12 @@ import org.apache.commons.lang3.builder.HashCodeBuilder;
import org.apache.commons.lang3.builder.ToStringBuilder;
import org.apache.commons.lang3.builder.ToStringStyle;
import static ca.uhn.fhir.jpa.model.util.SearchParamHash.hashSearchParam;
import static org.apache.commons.lang3.StringUtils.defaultString;
// @formatter:off
@Embeddable
@EntityListeners(IndexStorageOptimizationListener.class)
@Entity
@Table(
name = "HFJ_SPIDX_STRING",
@ -97,11 +101,6 @@ public class ResourceIndexedSearchParamString extends BaseResourceIndexedSearchP
*/
@Column(name = "HASH_NORM_PREFIX", nullable = true)
private Long myHashNormalizedPrefix;
/**
* @since 3.6.0 - At some point this should be made not-null
*/
@Column(name = "HASH_IDENTITY", nullable = true)
private Long myHashIdentity;
/**
* @since 3.4.0 - At some point this should be made not-null
*/
@ -180,24 +179,15 @@ public class ResourceIndexedSearchParamString extends BaseResourceIndexedSearchP
}
ResourceIndexedSearchParamString obj = (ResourceIndexedSearchParamString) theObj;
EqualsBuilder b = new EqualsBuilder();
b.append(getResourceType(), obj.getResourceType());
b.append(getParamName(), obj.getParamName());
b.append(getValueExact(), obj.getValueExact());
b.append(getHashIdentity(), obj.getHashIdentity());
b.append(getHashExact(), obj.getHashExact());
b.append(getHashNormalizedPrefix(), obj.getHashNormalizedPrefix());
b.append(getValueNormalized(), obj.getValueNormalized());
b.append(isMissing(), obj.isMissing());
return b.isEquals();
}
private Long getHashIdentity() {
return myHashIdentity;
}
public void setHashIdentity(Long theHashIdentity) {
myHashIdentity = theHashIdentity;
}
public Long getHashExact() {
return myHashExact;
}
@ -251,13 +241,12 @@ public class ResourceIndexedSearchParamString extends BaseResourceIndexedSearchP
@Override
public int hashCode() {
HashCodeBuilder b = new HashCodeBuilder();
b.append(getResourceType());
b.append(getParamName());
b.append(getValueExact());
b.append(getHashIdentity());
b.append(getHashExact());
b.append(getHashNormalizedPrefix());
b.append(getValueNormalized());
b.append(isMissing());
return b.toHashCode();
}
@ -306,7 +295,8 @@ public class ResourceIndexedSearchParamString extends BaseResourceIndexedSearchP
String theResourceType,
String theParamName,
String theValueExact) {
return hash(thePartitionSettings, theRequestPartitionId, theResourceType, theParamName, theValueExact);
return hashSearchParam(
thePartitionSettings, theRequestPartitionId, theResourceType, theParamName, theValueExact);
}
public static long calculateHashNormalized(
@ -345,7 +335,7 @@ public class ResourceIndexedSearchParamString extends BaseResourceIndexedSearchP
}
String value = StringUtil.left(theValueNormalized, hashPrefixLength);
return hash(thePartitionSettings, theRequestPartitionId, theResourceType, theParamName, value);
return hashSearchParam(thePartitionSettings, theRequestPartitionId, theResourceType, theParamName, value);
}
@Override

26
hapi-fhir-jpaserver-model/src/main/java/ca/uhn/fhir/jpa/model/entity/ResourceIndexedSearchParamToken.java
View File

@ -21,12 +21,14 @@ package ca.uhn.fhir.jpa.model.entity;
import ca.uhn.fhir.interceptor.model.RequestPartitionId;
import ca.uhn.fhir.jpa.model.config.PartitionSettings;
import ca.uhn.fhir.jpa.model.listener.IndexStorageOptimizationListener;
import ca.uhn.fhir.model.api.IQueryParameterType;
import ca.uhn.fhir.rest.api.Constants;
import ca.uhn.fhir.rest.param.TokenParam;
import jakarta.persistence.Column;
import jakarta.persistence.Embeddable;
import jakarta.persistence.Entity;
import jakarta.persistence.EntityListeners;
import jakarta.persistence.FetchType;
import jakarta.persistence.ForeignKey;
import jakarta.persistence.GeneratedValue;
@ -46,10 +48,12 @@ import org.apache.commons.lang3.builder.ToStringBuilder;
import org.apache.commons.lang3.builder.ToStringStyle;
import org.hibernate.search.mapper.pojo.mapping.definition.annotation.FullTextField;
import static ca.uhn.fhir.jpa.model.util.SearchParamHash.hashSearchParam;
import static org.apache.commons.lang3.StringUtils.defaultString;
import static org.apache.commons.lang3.StringUtils.trim;
@Embeddable
@EntityListeners(IndexStorageOptimizationListener.class)
@Entity
@Table(
name = "HFJ_SPIDX_TOKEN",
@ -89,11 +93,6 @@ public class ResourceIndexedSearchParamToken extends BaseResourceIndexedSearchPa
@GeneratedValue(strategy = GenerationType.AUTO, generator = "SEQ_SPIDX_TOKEN")
@Column(name = "SP_ID")
private Long myId;
/**
* @since 3.4.0 - At some point this should be made not-null
*/
@Column(name = "HASH_IDENTITY", nullable = true)
private Long myHashIdentity;
/**
* @since 3.4.0 - At some point this should be made not-null
*/
@ -217,9 +216,11 @@ public class ResourceIndexedSearchParamToken extends BaseResourceIndexedSearchPa
}
ResourceIndexedSearchParamToken obj = (ResourceIndexedSearchParamToken) theObj;
EqualsBuilder b = new EqualsBuilder();
b.append(getHashIdentity(), obj.getHashIdentity());
b.append(getHashSystem(), obj.getHashSystem());
b.append(getHashValue(), obj.getHashValue());
b.append(getHashSystemAndValue(), obj.getHashSystemAndValue());
b.append(isMissing(), obj.isMissing());
return b.isEquals();
}
@ -231,10 +232,6 @@ public class ResourceIndexedSearchParamToken extends BaseResourceIndexedSearchPa
myHashSystem = theHashSystem;
}
private void setHashIdentity(Long theHashIdentity) {
myHashIdentity = theHashIdentity;
}
public Long getHashSystemAndValue() {
return myHashSystemAndValue;
}
@ -283,11 +280,11 @@ public class ResourceIndexedSearchParamToken extends BaseResourceIndexedSearchPa
@Override
public int hashCode() {
HashCodeBuilder b = new HashCodeBuilder();
b.append(getResourceType());
b.append(getHashIdentity());
b.append(getHashValue());
b.append(getHashSystem());
b.append(getHashSystemAndValue());
b.append(isMissing());
return b.toHashCode();
}
@ -362,7 +359,8 @@ public class ResourceIndexedSearchParamToken extends BaseResourceIndexedSearchPa
String theResourceType,
String theParamName,
String theSystem) {
return hash(thePartitionSettings, theRequestPartitionId, theResourceType, theParamName, trim(theSystem));
return hashSearchParam(
thePartitionSettings, theRequestPartitionId, theResourceType, theParamName, trim(theSystem));
}
public static long calculateHashSystemAndValue(
@ -384,7 +382,7 @@ public class ResourceIndexedSearchParamToken extends BaseResourceIndexedSearchPa
String theParamName,
String theSystem,
String theValue) {
return hash(
return hashSearchParam(
thePartitionSettings,
theRequestPartitionId,
theResourceType,
@ -410,7 +408,7 @@ public class ResourceIndexedSearchParamToken extends BaseResourceIndexedSearchPa
String theParamName,
String theValue) {
String value = trim(theValue);
return hash(thePartitionSettings, theRequestPartitionId, theResourceType, theParamName, value);
return hashSearchParam(thePartitionSettings, theRequestPartitionId, theResourceType, theParamName, value);
}
@Override

25
hapi-fhir-jpaserver-model/src/main/java/ca/uhn/fhir/jpa/model/entity/ResourceIndexedSearchParamUri.java
View File

@ -21,11 +21,13 @@ package ca.uhn.fhir.jpa.model.entity;
import ca.uhn.fhir.interceptor.model.RequestPartitionId;
import ca.uhn.fhir.jpa.model.config.PartitionSettings;
import ca.uhn.fhir.jpa.model.listener.IndexStorageOptimizationListener;
import ca.uhn.fhir.model.api.IQueryParameterType;
import ca.uhn.fhir.rest.param.UriParam;
import jakarta.persistence.Column;
import jakarta.persistence.Embeddable;
import jakarta.persistence.Entity;
import jakarta.persistence.EntityListeners;
import jakarta.persistence.FetchType;
import jakarta.persistence.ForeignKey;
import jakarta.persistence.GeneratedValue;
@ -42,9 +44,11 @@ import org.apache.commons.lang3.builder.HashCodeBuilder;
import org.apache.commons.lang3.builder.ToStringBuilder;
import org.hibernate.search.mapper.pojo.mapping.definition.annotation.FullTextField;
import static ca.uhn.fhir.jpa.model.util.SearchParamHash.hashSearchParam;
import static org.apache.commons.lang3.StringUtils.defaultString;
@Embeddable
@EntityListeners(IndexStorageOptimizationListener.class)
@Entity
@Table(
name = "HFJ_SPIDX_URI",
@ -84,11 +88,6 @@ public class ResourceIndexedSearchParamUri extends BaseResourceIndexedSearchPara
*/
@Column(name = "HASH_URI", nullable = true)
private Long myHashUri;
/**
* @since 3.5.0 - At some point this should be made not-null
*/
@Column(name = "HASH_IDENTITY", nullable = true)
private Long myHashIdentity;
@ManyToOne(
optional = false,
@ -161,22 +160,13 @@ public class ResourceIndexedSearchParamUri extends BaseResourceIndexedSearchPara
}
ResourceIndexedSearchParamUri obj = (ResourceIndexedSearchParamUri) theObj;
EqualsBuilder b = new EqualsBuilder();
b.append(getResourceType(), obj.getResourceType());
b.append(getParamName(), obj.getParamName());
b.append(getUri(), obj.getUri());
b.append(getHashUri(), obj.getHashUri());
b.append(getHashIdentity(), obj.getHashIdentity());
b.append(isMissing(), obj.isMissing());
return b.isEquals();
}
private Long getHashIdentity() {
return myHashIdentity;
}
private void setHashIdentity(long theHashIdentity) {
myHashIdentity = theHashIdentity;
}
public Long getHashUri() {
return myHashUri;
}
@ -207,11 +197,10 @@ public class ResourceIndexedSearchParamUri extends BaseResourceIndexedSearchPara
@Override
public int hashCode() {
HashCodeBuilder b = new HashCodeBuilder();
b.append(getResourceType());
b.append(getParamName());
b.append(getUri());
b.append(getHashUri());
b.append(getHashIdentity());
b.append(isMissing());
return b.toHashCode();
}
@ -257,7 +246,7 @@ public class ResourceIndexedSearchParamUri extends BaseResourceIndexedSearchPara
String theResourceType,
String theParamName,
String theUri) {
return hash(thePartitionSettings, theRequestPartitionId, theResourceType, theParamName, theUri);
return hashSearchParam(thePartitionSettings, theRequestPartitionId, theResourceType, theParamName, theUri);
}
@Override

5
hapi-fhir-jpaserver-model/src/main/java/ca/uhn/fhir/jpa/model/entity/SearchParamPresentEntity.java
View File

@ -42,6 +42,8 @@ import org.apache.commons.lang3.builder.ToStringStyle;
import java.io.Serializable;
import static ca.uhn.fhir.jpa.model.util.SearchParamHash.hashSearchParam;
@Entity
@Table(
name = "HFJ_RES_PARAM_PRESENT",
@ -212,7 +214,6 @@ public class SearchParamPresentEntity extends BasePartitionable implements Seria
String theParamName,
Boolean thePresent) {
String string = thePresent != null ? Boolean.toString(thePresent) : Boolean.toString(false);
return BaseResourceIndexedSearchParam.hash(
thePartitionSettings, theRequestPartitionId, theResourceType, theParamName, string);
return hashSearchParam(thePartitionSettings, theRequestPartitionId, theResourceType, theParamName, string);
}
}

61
hapi-fhir-jpaserver-model/src/main/java/ca/uhn/fhir/jpa/model/entity/StorageSettings.java
View File

@ -21,6 +21,7 @@ package ca.uhn.fhir.jpa.model.entity;
import ca.uhn.fhir.context.ParserOptions;
import ca.uhn.fhir.i18n.Msg;
import ca.uhn.fhir.jpa.model.config.PartitionSettings;
import ca.uhn.fhir.jpa.util.ISequenceValueMassager;
import ca.uhn.fhir.model.api.TemporalPrecisionEnum;
import ca.uhn.fhir.rest.server.interceptor.ResponseTerminologyTranslationSvc;
@ -134,6 +135,14 @@ public class StorageSettings {
*/
private boolean myValidateResourceStatusForPackageUpload = true;
/**
* If set to <code>true</code>, the server will not write data to the <code>SP_NAME, RES_TYPE, SP_UPDATED</code>
* columns for all HFJ_SPIDX tables.
*
* @since 7.4.0
*/
private boolean myIndexStorageOptimized = false;
/**
* Constructor
*/
@ -277,6 +286,58 @@ public class StorageSettings {
myIndexMissingFieldsEnabled = theIndexMissingFields;
}
/**
* If set to <code>true</code> (default is false), the server will not write data
* to the <code>SP_NAME, RES_TYPE, SP_UPDATED</code> columns for all HFJ_SPIDX tables.
* <p>
* This feature may be enabled on servers where HFJ_SPIDX tables are expected
* to have a large amount of data (millions of rows) in order to reduce overall storage size.
* </p>
* <p>
* Note that this setting only applies to newly inserted and updated rows in HFJ_SPIDX tables.
* In order to apply this optimization setting to existing HFJ_SPIDX index rows,
* <code>$reindex</code> operation should be executed at the instance or server level.
* <p>
* <p>
* If this setting is enabled, {@link PartitionSettings#isIncludePartitionInSearchHashes()} should be disabled.
* </p>
* <p>
* If {@link StorageSettings#getIndexMissingFields()} is enabled, the following index may need to be added
* into the HFJ_SPIDX tables to improve the search performance: <code>HASH_IDENTITY, SP_MISSING, RES_ID, PARTITION_ID</code>
* </p>
*
* @since 7.4.0
*/
public boolean isIndexStorageOptimized() {
return myIndexStorageOptimized;
}
/**
* If set to <code>true</code> (default is false), the server will not write data
* to the <code>SP_NAME, RES_TYPE, SP_UPDATED</code> columns for all HFJ_SPIDX tables.
* <p>
* This feature may be enabled on servers where HFJ_SPIDX tables are expected
* to have a large amount of data (millions of rows) in order to reduce overall storage size.
* </p>
* <p>
* Note that this setting only applies to newly inserted and updated rows in HFJ_SPIDX tables.
* In order to apply this optimization setting to existing HFJ_SPIDX index rows,
* <code>$reindex</code> operation should be executed at the instance or server level.
* <p>
* <p>
* If this setting is enabled, {@link PartitionSettings#isIncludePartitionInSearchHashes()} should be set to <code>false</code>.
* </p>
* <p>
* If {@link StorageSettings#getIndexMissingFields()} ()} is enabled, the following index may need to be added
* into the HFJ_SPIDX tables to improve the search performance: <code>HASH_IDENTITY, SP_MISSING, RES_ID, PARTITION_ID</code>
* </p>
*
* @since 7.4.0
*/
public void setIndexStorageOptimized(boolean theIndexStorageOptimized) {
myIndexStorageOptimized = theIndexStorageOptimized;
}
/**
* If this is enabled (disabled by default), Mass Ingestion Mode is enabled. In this mode, a number of
* runtime checks are disabled. This mode is designed for rapid backloading of data while the system is not

99
hapi-fhir-jpaserver-model/src/main/java/ca/uhn/fhir/jpa/model/listener/IndexStorageOptimizationListener.java Normal file
View File

@ -0,0 +1,99 @@
/*
* #%L
* HAPI FHIR JPA Model
* %%
* Copyright (C) 2014 - 2024 Smile CDR, Inc.
* %%
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
* #L%
*/
package ca.uhn.fhir.jpa.model.listener;
import ca.uhn.fhir.jpa.model.entity.BaseResourceIndexedSearchParam;
import ca.uhn.fhir.jpa.model.entity.StorageSettings;
import ca.uhn.fhir.jpa.model.search.ISearchParamHashIdentityRegistry;
import ca.uhn.fhir.rest.server.util.IndexedSearchParam;
import jakarta.persistence.PostLoad;
import jakarta.persistence.PostPersist;
import jakarta.persistence.PostUpdate;
import jakarta.persistence.PrePersist;
import jakarta.persistence.PreUpdate;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.ApplicationContext;
import java.util.Optional;
/**
* Sets <code>SP_NAME, RES_TYPE, SP_UPDATED</code> column values to null for all HFJ_SPIDX tables
* if storage setting {@link ca.uhn.fhir.jpa.model.entity.StorageSettings#isIndexStorageOptimized()} is enabled.
* <p>
* Using EntityListener to change HFJ_SPIDX column values right before insert/update to database.
* </p>
* <p>
* As <code>SP_NAME, RES_TYPE</code> values could still be used after merge/persist to database, we are restoring
* them from <code>HASH_IDENTITY</code> value.
*</p>
* See {@link ca.uhn.fhir.jpa.model.entity.StorageSettings#setIndexStorageOptimized(boolean)}
*/
public class IndexStorageOptimizationListener {
public IndexStorageOptimizationListener(
@Autowired StorageSettings theStorageSettings, @Autowired ApplicationContext theApplicationContext) {
this.myStorageSettings = theStorageSettings;
this.myApplicationContext = theApplicationContext;
}
private final StorageSettings myStorageSettings;
private final ApplicationContext myApplicationContext;
@PrePersist
@PreUpdate
public void optimizeSearchParams(Object theEntity) {
if (myStorageSettings.isIndexStorageOptimized() && theEntity instanceof BaseResourceIndexedSearchParam) {
((BaseResourceIndexedSearchParam) theEntity).optimizeIndexStorage();
}
}
@PostLoad
@PostPersist
@PostUpdate
public void restoreSearchParams(Object theEntity) {
if (myStorageSettings.isIndexStorageOptimized() && theEntity instanceof BaseResourceIndexedSearchParam) {
restoreSearchParams((BaseResourceIndexedSearchParam) theEntity);
}
}
/**
* As <code>SP_NAME, RES_TYPE</code> values could still be used after merge/persist to database (mostly by tests),
* we are restoring them from <code>HASH_IDENTITY</code> value.
* Note that <code>SP_NAME, RES_TYPE</code> values are not recovered if
* {@link ca.uhn.fhir.jpa.model.entity.StorageSettings#isIndexOnContainedResources()} or
* {@link ca.uhn.fhir.jpa.model.entity.StorageSettings#isIndexOnContainedResourcesRecursively()}
* settings are enabled.
*/
private void restoreSearchParams(BaseResourceIndexedSearchParam theResourceIndexedSearchParam) {
// getting ISearchParamHashIdentityRegistry from the App Context as it is initialized after EntityListeners
ISearchParamHashIdentityRegistry searchParamRegistry =
myApplicationContext.getBean(ISearchParamHashIdentityRegistry.class);
Optional<IndexedSearchParam> indexedSearchParamOptional =
searchParamRegistry.getIndexedSearchParamByHashIdentity(
theResourceIndexedSearchParam.getHashIdentity());
if (indexedSearchParamOptional.isPresent()) {
theResourceIndexedSearchParam.setResourceType(
indexedSearchParamOptional.get().getResourceType());
theResourceIndexedSearchParam.restoreParamName(
indexedSearchParamOptional.get().getParameterName());
}
}
}

9
hapi-fhir-jpaserver-model/src/main/java/ca/uhn/fhir/jpa/model/search/ISearchParamHashIdentityRegistry.java Normal file
View File

@ -0,0 +1,9 @@
package ca.uhn.fhir.jpa.model.search;
import ca.uhn.fhir.rest.server.util.IndexedSearchParam;
import java.util.Optional;
public interface ISearchParamHashIdentityRegistry {
Optional<IndexedSearchParam> getIndexedSearchParamByHashIdentity(Long theHashIdentity);
}

85
hapi-fhir-jpaserver-model/src/main/java/ca/uhn/fhir/jpa/model/util/SearchParamHash.java Normal file
View File

@ -0,0 +1,85 @@
/*-
* #%L
* HAPI FHIR JPA Model
* %%
* Copyright (C) 2014 - 2024 Smile CDR, Inc.
* %%
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
* #L%
*/
package ca.uhn.fhir.jpa.model.util;
import ca.uhn.fhir.i18n.Msg;
import ca.uhn.fhir.interceptor.model.RequestPartitionId;
import ca.uhn.fhir.jpa.model.config.PartitionSettings;
import ca.uhn.fhir.rest.server.exceptions.InternalErrorException;
import ca.uhn.fhir.util.UrlUtil;
import com.google.common.base.Charsets;
import com.google.common.hash.HashCode;
import com.google.common.hash.HashFunction;
import com.google.common.hash.Hasher;
import com.google.common.hash.Hashing;
/**
* Utility class for calculating hashes of SearchParam entity fields.
*/
public class SearchParamHash {
/**
* Don't change this without careful consideration. You will break existing hashes!
*/
private static final HashFunction HASH_FUNCTION = Hashing.murmur3_128(0);
/**
* Don't make this public 'cause nobody better be able to modify it!
*/
private static final byte[] DELIMITER_BYTES = "|".getBytes(Charsets.UTF_8);
private SearchParamHash() {}
/**
* Applies a fast and consistent hashing algorithm to a set of strings
*/
public static long hashSearchParam(
PartitionSettings thePartitionSettings, RequestPartitionId theRequestPartitionId, String... theValues) {
Hasher hasher = HASH_FUNCTION.newHasher();
if (thePartitionSettings.isPartitioningEnabled()
&& thePartitionSettings.isIncludePartitionInSearchHashes()
&& theRequestPartitionId != null) {
if (theRequestPartitionId.getPartitionIds().size() > 1) {
throw new InternalErrorException(Msg.code(1527)
+ "Can not search multiple partitions when partitions are included in search hashes");
}
Integer partitionId = theRequestPartitionId.getFirstPartitionIdOrNull();
if (partitionId != null) {
hasher.putInt(partitionId);
}
}
for (String next : theValues) {
if (next == null) {
hasher.putByte((byte) 0);
} else {
next = UrlUtil.escapeUrlParam(next);
byte[] bytes = next.getBytes(Charsets.UTF_8);
hasher.putBytes(bytes);
}
hasher.putBytes(DELIMITER_BYTES);
}
HashCode hashCode = hasher.hash();
long retVal = hashCode.asLong();
return retVal;
}
}

60
hapi-fhir-jpaserver-model/src/test/java/ca/uhn/fhir/jpa/model/entity/ResourceIndexedSearchParamCoordsTest.java
View File

@ -2,15 +2,16 @@ package ca.uhn.fhir.jpa.model.entity;
import ca.uhn.fhir.jpa.model.config.PartitionSettings;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.CsvSource;
import static org.assertj.core.api.Assertions.assertThat;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertNotNull;
import static org.junit.jupiter.api.Assertions.assertNotEquals;
public class ResourceIndexedSearchParamCoordsTest {
@Test
public void testEquals() {
public void testEqualsAndHashCode_withSameParams_equalsIsTrueAndHashCodeIsSame() {
ResourceIndexedSearchParamCoords val1 = new ResourceIndexedSearchParamCoords()
.setLatitude(100)
.setLongitude(10);
@ -21,8 +22,55 @@ public class ResourceIndexedSearchParamCoordsTest {
.setLongitude(10);
val2.setPartitionSettings(new PartitionSettings());
val2.calculateHashes();
assertNotNull(val1);
assertEquals(val1, val2);
assertThat("").isNotEqualTo(val1);
validateEquals(val2, val1);
}
private void validateEquals(ResourceIndexedSearchParamCoords theParam1, ResourceIndexedSearchParamCoords theParam2) {
assertEquals(theParam2, theParam1);
assertEquals(theParam1, theParam2);
assertEquals(theParam1.hashCode(), theParam2.hashCode());
}
@Test
public void testEqualsAndHashCode_withOptimizedSearchParam_equalsIsTrueAndHashCodeIsSame() {
ResourceIndexedSearchParamCoords param = new ResourceIndexedSearchParamCoords(
new PartitionSettings(), "Patient", "param", 100, 10);
ResourceIndexedSearchParamCoords param2 = new ResourceIndexedSearchParamCoords(
new PartitionSettings(), "Patient", "param", 100, 10);
param2.optimizeIndexStorage();
validateEquals(param, param2);
}
@ParameterizedTest
@CsvSource({
"Patient, param, 100, 100, false, Observation, param, 100, 100, false, ResourceType is different",
"Patient, param, 100, 100, false, Patient, name, 100, 100, false, ParamName is different",
"Patient, param, 10, 100, false, Patient, param, 100, 100, false, Latitude is different",
"Patient, param, 100, 10, false, Patient, param, 100, 100, false, Longitude is different",
"Patient, param, 100, 100, true, Patient, param, 100, 100, false, Missing is different",
})
public void testEqualsAndHashCode_withDifferentParams_equalsIsFalseAndHashCodeIsDifferent(String theFirstResourceType,
String theFirstParamName,
double theFirstLatitude,
double theFirstLongitude,
boolean theFirstMissing,
String theSecondResourceType,
String theSecondParamName,
double theSecondLatitude,
double theSecondLongitude,
boolean theSecondMissing,
String theMessage) {
ResourceIndexedSearchParamCoords param = new ResourceIndexedSearchParamCoords(
new PartitionSettings(), theFirstResourceType, theFirstParamName, theFirstLatitude, theFirstLongitude);
param.setMissing(theFirstMissing);
ResourceIndexedSearchParamCoords param2 = new ResourceIndexedSearchParamCoords(
new PartitionSettings(), theSecondResourceType, theSecondParamName, theSecondLatitude, theSecondLongitude);
param2.setMissing(theSecondMissing);
assertNotEquals(param, param2, theMessage);
assertNotEquals(param2, param, theMessage);
assertNotEquals(param.hashCode(), param2.hashCode(), theMessage);
}
}

108
hapi-fhir-jpaserver-model/src/test/java/ca/uhn/fhir/jpa/model/entity/ResourceIndexedSearchParamDateTest.java
View File

@ -3,16 +3,17 @@ package ca.uhn.fhir.jpa.model.entity;
import ca.uhn.fhir.jpa.model.config.PartitionSettings;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.CsvSource;
import java.sql.Timestamp;
import java.time.Instant;
import java.util.Calendar;
import java.util.Date;
import static org.assertj.core.api.Assertions.assertThat;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertNotNull;
import static org.junit.jupiter.api.Assertions.assertTrue;
import static org.junit.jupiter.api.Assertions.assertNotEquals;
public class ResourceIndexedSearchParamDateTest {
@ -43,9 +44,7 @@ public class ResourceIndexedSearchParamDateTest {
ResourceIndexedSearchParamDate param = new ResourceIndexedSearchParamDate(new PartitionSettings(), "Patient", "SomeResource", null, null, null, null, "SomeValue");
ResourceIndexedSearchParamDate param2 = new ResourceIndexedSearchParamDate(new PartitionSettings(), "Patient", "SomeResource", null, null, null, null, "SomeValue");
assertTrue(param.equals(param2));
assertTrue(param2.equals(param));
assertEquals(param.hashCode(), param2.hashCode());
validateEquals(param, param2);
}
@Test
@ -53,9 +52,7 @@ public class ResourceIndexedSearchParamDateTest {
ResourceIndexedSearchParamDate param = new ResourceIndexedSearchParamDate(new PartitionSettings(), "Patient", "SomeResource", date1A, null, date2A, null, "SomeValue");
ResourceIndexedSearchParamDate param2 = new ResourceIndexedSearchParamDate(new PartitionSettings(), "Patient", "SomeResource", date1B, null, date2B, null, "SomeValue");
assertTrue(param.equals(param2));
assertTrue(param2.equals(param));
assertEquals(param.hashCode(), param2.hashCode());
validateEquals(param, param2);
}
@Test
@ -63,9 +60,7 @@ public class ResourceIndexedSearchParamDateTest {
ResourceIndexedSearchParamDate param = new ResourceIndexedSearchParamDate(new PartitionSettings(), "Patient", "SomeResource", timestamp1A, null, timestamp2A, null, "SomeValue");
ResourceIndexedSearchParamDate param2 = new ResourceIndexedSearchParamDate(new PartitionSettings(), "Patient", "SomeResource", timestamp1B, null, timestamp2B, null, "SomeValue");
assertTrue(param.equals(param2));
assertTrue(param2.equals(param));
assertEquals(param.hashCode(), param2.hashCode());
validateEquals(param, param2);
}
// Scenario that occurs when updating a resource with a date search parameter. One date will be a java.util.Date, the
@ -75,9 +70,23 @@ public class ResourceIndexedSearchParamDateTest {
ResourceIndexedSearchParamDate param = new ResourceIndexedSearchParamDate(new PartitionSettings(), "Patient", "SomeResource", date1A, null, date2A, null, "SomeValue");
ResourceIndexedSearchParamDate param2 = new ResourceIndexedSearchParamDate(new PartitionSettings(), "Patient", "SomeResource", timestamp1A, null, timestamp2A, null, "SomeValue");
assertTrue(param.equals(param2));
assertTrue(param2.equals(param));
assertEquals(param.hashCode(), param2.hashCode());
validateEquals(param, param2);
}
@Test
public void equalsIsTrueForOptimizedSearchParam() {
ResourceIndexedSearchParamDate param = new ResourceIndexedSearchParamDate(new PartitionSettings(), "Patient", "SomeResource", date1A, null, date2A, null, "SomeValue");
ResourceIndexedSearchParamDate param2 = new ResourceIndexedSearchParamDate(new PartitionSettings(), "Patient", "SomeResource", date1A, null, date2A, null, "SomeValue");
param2.optimizeIndexStorage();
validateEquals(param, param2);
}
private void validateEquals(ResourceIndexedSearchParamDate theParam, ResourceIndexedSearchParamDate theParam2) {
assertEquals(theParam, theParam2);
assertEquals(theParam2, theParam);
assertEquals(theParam.hashCode(), theParam2.hashCode());
}
@Test
@ -85,9 +94,7 @@ public class ResourceIndexedSearchParamDateTest {
ResourceIndexedSearchParamDate param = new ResourceIndexedSearchParamDate(new PartitionSettings(), "Patient", "SomeResource", date1A, null, date2A, null, "SomeValue");
ResourceIndexedSearchParamDate param2 = new ResourceIndexedSearchParamDate(new PartitionSettings(), "Patient", "SomeResource", date2A, null, date1A, null, "SomeValue");
assertFalse(param.equals(param2));
assertFalse(param2.equals(param));
assertThat(param2.hashCode()).isNotEqualTo(param.hashCode());
validateNotEquals(param, param2);
}
@Test
@ -95,9 +102,7 @@ public class ResourceIndexedSearchParamDateTest {
ResourceIndexedSearchParamDate param = new ResourceIndexedSearchParamDate(new PartitionSettings(), "Patient", "SomeResource", date1A, null, date2A, null, "SomeValue");
ResourceIndexedSearchParamDate param2 = new ResourceIndexedSearchParamDate(new PartitionSettings(), "Patient", "SomeResource", null, null, null, null, "SomeValue");
assertFalse(param.equals(param2));
assertFalse(param2.equals(param));
assertThat(param2.hashCode()).isNotEqualTo(param.hashCode());
validateNotEquals(param, param2);
}
@Test
@ -105,9 +110,7 @@ public class ResourceIndexedSearchParamDateTest {
ResourceIndexedSearchParamDate param = new ResourceIndexedSearchParamDate(new PartitionSettings(), "Patient", "SomeResource", timestamp1A, null, timestamp2A, null, "SomeValue");
ResourceIndexedSearchParamDate param2 = new ResourceIndexedSearchParamDate(new PartitionSettings(), "Patient", "SomeResource", timestamp2A, null, timestamp1A, null, "SomeValue");
assertFalse(param.equals(param2));
assertFalse(param2.equals(param));
assertThat(param2.hashCode()).isNotEqualTo(param.hashCode());
validateNotEquals(param, param2);
}
@Test
@ -115,14 +118,18 @@ public class ResourceIndexedSearchParamDateTest {
ResourceIndexedSearchParamDate param = new ResourceIndexedSearchParamDate(new PartitionSettings(), "Patient", "SomeResource", date1A, null, date2A, null, "SomeValue");
ResourceIndexedSearchParamDate param2 = new ResourceIndexedSearchParamDate(new PartitionSettings(), "Patient", "SomeResource", timestamp2A, null, timestamp1A, null, "SomeValue");
assertFalse(param.equals(param2));
assertFalse(param2.equals(param));
assertThat(param2.hashCode()).isNotEqualTo(param.hashCode());
validateNotEquals(param, param2);
}
private void validateNotEquals(ResourceIndexedSearchParamDate theParam, ResourceIndexedSearchParamDate theParam2) {
assertNotEquals(theParam, theParam2);
assertNotEquals(theParam2, theParam);
assertThat(theParam2.hashCode()).isNotEqualTo(theParam.hashCode());
}
@Test
public void testEquals() {
public void testEqualsAndHashCode_withSameParams_equalsIsTrueAndHashCodeIsSame() {
ResourceIndexedSearchParamDate val1 = new ResourceIndexedSearchParamDate()
.setValueHigh(new Date(100000000L))
.setValueLow(new Date(111111111L));
@ -133,8 +140,47 @@ public class ResourceIndexedSearchParamDateTest {
.setValueLow(new Date(111111111L));
val2.setPartitionSettings(new PartitionSettings());
val2.calculateHashes();
assertNotNull(val1);
assertEquals(val1, val2);
assertThat("").isNotEqualTo(val1);
validateEquals(val1, val2);
}
@ParameterizedTest
@CsvSource({
"Patient, param, 2018-04-25T14:05:15.953Z, 2019-04-25T14:05:15.953Z, false, " +
"Observation, param, 2018-04-25T14:05:15.953Z, 2019-04-25T14:05:15.953Z, false, ResourceType is different",
"Patient, param, 2018-04-25T14:05:15.953Z, 2019-04-25T14:05:15.953Z, false, " +
"Patient, name, 2018-04-25T14:05:15.953Z, 2019-04-25T14:05:15.953Z, false, ParamName is different",
"Patient, param, 2017-04-25T14:05:15.953Z, 2019-04-25T14:05:15.953Z, false, " +
"Patient, param, 2018-04-25T14:05:15.953Z, 2019-04-25T14:05:15.953Z, false, LowDate is different",
"Patient, param, 2018-04-25T14:05:15.953Z, 2019-04-25T14:05:15.953Z, false, " +
"Patient, param, 2018-04-25T14:05:15.953Z, 2020-04-25T14:05:15.953Z, false, HighDate is different",
"Patient, param, 2018-04-25T14:05:15.953Z, 2019-04-25T14:05:15.953Z, true, " +
"Patient, param, 2018-04-25T14:05:15.953Z, 2019-04-25T14:05:15.953Z, false, Missing is different",
})
public void testEqualsAndHashCode_withDifferentParams_equalsIsFalseAndHashCodeIsDifferent(String theFirstResourceType,
String theFirstParamName,
String theFirstLowDate,
String theFirstHighDate,
boolean theFirstMissing,
String theSecondResourceType,
String theSecondParamName,
String theSecondLowDate,
String theSecondHighDate,
boolean theSecondMissing,
String theMessage) {
Date firstLowDate = Date.from(Instant.parse(theFirstLowDate));
Date firstHighDate = Date.from(Instant.parse(theFirstHighDate));
ResourceIndexedSearchParamDate param = new ResourceIndexedSearchParamDate(new PartitionSettings(),
theFirstResourceType, theFirstParamName, firstLowDate, theFirstLowDate, firstHighDate, theFirstHighDate, null);
param.setMissing(theFirstMissing);
Date secondLowDate = Date.from(Instant.parse(theSecondLowDate));
Date secondHighDate = Date.from(Instant.parse(theSecondHighDate));
ResourceIndexedSearchParamDate param2 = new ResourceIndexedSearchParamDate(new PartitionSettings(),
theSecondResourceType, theSecondParamName, secondLowDate, theSecondLowDate, secondHighDate, theSecondHighDate, null);
param2.setMissing(theSecondMissing);
assertNotEquals(param, param2, theMessage);
assertNotEquals(param2, param, theMessage);
assertNotEquals(param.hashCode(), param2.hashCode(), theMessage);
}
}

49
hapi-fhir-jpaserver-model/src/test/java/ca/uhn/fhir/jpa/model/entity/ResourceIndexedSearchParamNumberTest.java
View File

@ -3,23 +3,29 @@ package ca.uhn.fhir.jpa.model.entity;
import ca.uhn.fhir.jpa.model.config.PartitionSettings;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.CsvSource;
import java.math.BigDecimal;
import static org.assertj.core.api.Assertions.assertThat;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertNotEquals;
public class ResourceIndexedSearchParamNumberTest {
private static final String GRITTSCORE = "grittscore";
public static final ResourceIndexedSearchParamNumber PARAM_VALUE_10_FIRST = new ResourceIndexedSearchParamNumber(new PartitionSettings(), "Patient", GRITTSCORE, BigDecimal.valueOf(10));
public static final ResourceIndexedSearchParamNumber PARAM_VALUE_10_SECOND = new ResourceIndexedSearchParamNumber(new PartitionSettings(), "Patient", GRITTSCORE, BigDecimal.valueOf(10));
public static final ResourceIndexedSearchParamNumber PARAM_VALUE_12_FIRST = new ResourceIndexedSearchParamNumber(new PartitionSettings(), "Patient", GRITTSCORE, BigDecimal.valueOf(12));
public static ResourceIndexedSearchParamNumber PARAM_VALUE_10_FIRST;
public static ResourceIndexedSearchParamNumber PARAM_VALUE_10_SECOND;
public static ResourceIndexedSearchParamNumber PARAM_VALUE_12_FIRST;
@BeforeEach
void setUp() {
final ResourceTable resourceTable = new ResourceTable();
resourceTable.setId(1L);
PARAM_VALUE_10_FIRST = new ResourceIndexedSearchParamNumber(new PartitionSettings(), "Patient", GRITTSCORE, BigDecimal.valueOf(10));
PARAM_VALUE_10_SECOND = new ResourceIndexedSearchParamNumber(new PartitionSettings(), "Patient", GRITTSCORE, BigDecimal.valueOf(10));
PARAM_VALUE_12_FIRST = new ResourceIndexedSearchParamNumber(new PartitionSettings(), "Patient", GRITTSCORE, BigDecimal.valueOf(12));
PARAM_VALUE_10_FIRST.setResource(resourceTable);
PARAM_VALUE_10_SECOND.setResource(resourceTable);
PARAM_VALUE_12_FIRST.setResource(resourceTable);
@ -32,6 +38,34 @@ public class ResourceIndexedSearchParamNumberTest {
assertThat(PARAM_VALUE_12_FIRST.hashCode()).isNotEqualTo(PARAM_VALUE_10_FIRST.hashCode());
}
@ParameterizedTest
@CsvSource({
"Patient, param, 10, false, Observation, param, 10, false, ResourceType is different",
"Patient, param, 10, false, Patient, name, 10, false, ParamName is different",
"Patient, param, 10, false, Patient, param, 9, false, Value is different",
"Patient, param, 10, false, Patient, param, 10, true, Missing is different",
})
public void testEqualsAndHashCode_withDifferentParams_equalsIsFalseAndHashCodeIsDifferent(String theFirstResourceType,
String theFirstParamName,
int theFirstValue,
boolean theFirstMissing,
String theSecondResourceType,
String theSecondParamName,
int theSecondValue,
boolean theSecondMissing,
String theMessage) {
ResourceIndexedSearchParamNumber param = new ResourceIndexedSearchParamNumber(
new PartitionSettings(), theFirstResourceType, theFirstParamName, BigDecimal.valueOf(theFirstValue));
param.setMissing(theFirstMissing);
ResourceIndexedSearchParamNumber param2 = new ResourceIndexedSearchParamNumber(
new PartitionSettings(), theSecondResourceType, theSecondParamName, BigDecimal.valueOf(theSecondValue));
param2.setMissing(theSecondMissing);
assertNotEquals(param, param2, theMessage);
assertNotEquals(param2, param, theMessage);
assertNotEquals(param.hashCode(), param2.hashCode(), theMessage);
}
@Test
void equalByReference() {
assertEquals(PARAM_VALUE_10_FIRST, PARAM_VALUE_10_FIRST);
@ -44,4 +78,13 @@ public class ResourceIndexedSearchParamNumberTest {
assertEquals(PARAM_VALUE_10_SECOND, PARAM_VALUE_10_FIRST);
assertEquals(PARAM_VALUE_10_FIRST.hashCode(), PARAM_VALUE_10_SECOND.hashCode());
}
@Test
void equalsIsTrueForOptimizedSearchParam() {
PARAM_VALUE_10_SECOND.optimizeIndexStorage();
assertEquals(PARAM_VALUE_10_FIRST, PARAM_VALUE_10_SECOND);
assertEquals(PARAM_VALUE_10_SECOND, PARAM_VALUE_10_FIRST);
assertEquals(PARAM_VALUE_10_FIRST.hashCode(), PARAM_VALUE_10_SECOND.hashCode());
}
}

60
hapi-fhir-jpaserver-model/src/test/java/ca/uhn/fhir/jpa/model/entity/ResourceIndexedSearchParamQuantityNormalizedTest.java
View File

@ -2,10 +2,11 @@ package ca.uhn.fhir.jpa.model.entity;
import ca.uhn.fhir.jpa.model.config.PartitionSettings;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.CsvSource;
import static org.assertj.core.api.Assertions.assertThat;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertNotNull;
import static org.junit.jupiter.api.Assertions.assertNotEquals;
public class ResourceIndexedSearchParamQuantityNormalizedTest {
@ -20,10 +21,59 @@ public class ResourceIndexedSearchParamQuantityNormalizedTest {
.setValue(Double.parseDouble("123"));
val2.setPartitionSettings(new PartitionSettings());
val2.calculateHashes();
assertNotNull(val1);
assertEquals(val1, val2);
assertThat("").isNotEqualTo(val1);
validateEquals(val1, val2);
}
@Test
public void equalsIsTrueForOptimizedSearchParam() {
BaseResourceIndexedSearchParamQuantity param = new ResourceIndexedSearchParamQuantityNormalized(
new PartitionSettings(), "Patient", "param", 123.0, "http://unitsofmeasure.org", "kg");
BaseResourceIndexedSearchParamQuantity param2 = new ResourceIndexedSearchParamQuantityNormalized(
new PartitionSettings(), "Patient", "param", 123.0, "http://unitsofmeasure.org", "kg");
param2.optimizeIndexStorage();
validateEquals(param, param2);
}
private void validateEquals(BaseResourceIndexedSearchParamQuantity theParam1,
BaseResourceIndexedSearchParamQuantity theParam2) {
assertEquals(theParam2, theParam1);
assertEquals(theParam1, theParam2);
assertEquals(theParam1.hashCode(), theParam2.hashCode());
}
@ParameterizedTest
@CsvSource({
"Patient, param, 123.0, units, kg, false, Observation, param, 123.0, units, kg, false, ResourceType is different",
"Patient, param, 123.0, units, kg, false, Patient, name, 123.0, units, kg, false, ParamName is different",
"Patient, param, 123.0, units, kg, false, Patient, param, 321.0, units, kg, false, Value is different",
"Patient, param, 123.0, units, kg, false, Patient, param, 123.0, unitsDiff, kg, false, System is different",
"Patient, param, 123.0, units, kg, false, Patient, param, 123.0, units, lb, false, Units is different",
"Patient, param, 123.0, units, kg, false, Patient, param, 123.0, units, kg, true, Missing is different",
})
public void testEqualsAndHashCode_withDifferentParams_equalsIsFalseAndHashCodeIsDifferent(String theFirstResourceType,
String theFirstParamName,
double theFirstValue,
String theFirstSystem,
String theFirstUnits,
boolean theFirstMissing,
String theSecondResourceType,
String theSecondParamName,
double theSecondValue,
String theSecondSystem,
String theSecondUnits,
boolean theSecondMissing,
String theMessage) {
BaseResourceIndexedSearchParamQuantity param = new ResourceIndexedSearchParamQuantityNormalized(
new PartitionSettings(), theFirstResourceType, theFirstParamName, theFirstValue, theFirstSystem, theFirstUnits);
param.setMissing(theFirstMissing);
BaseResourceIndexedSearchParamQuantity param2 = new ResourceIndexedSearchParamQuantityNormalized(
new PartitionSettings(), theSecondResourceType, theSecondParamName, theSecondValue, theSecondSystem, theSecondUnits);
param2.setMissing(theSecondMissing);
assertNotEquals(param, param2, theMessage);
assertNotEquals(param2, param, theMessage);
assertNotEquals(param.hashCode(), param2.hashCode(), theMessage);
}
}

59
hapi-fhir-jpaserver-model/src/test/java/ca/uhn/fhir/jpa/model/entity/ResourceIndexedSearchParamQuantityTest.java
View File

@ -2,12 +2,13 @@ package ca.uhn.fhir.jpa.model.entity;
import ca.uhn.fhir.jpa.model.config.PartitionSettings;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.CsvSource;
import java.math.BigDecimal;
import static org.assertj.core.api.Assertions.assertThat;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertNotNull;
import static org.junit.jupiter.api.Assertions.assertNotEquals;
public class ResourceIndexedSearchParamQuantityTest {
@ -38,10 +39,58 @@ public class ResourceIndexedSearchParamQuantityTest {
.setValue(new BigDecimal(123));
val2.setPartitionSettings(new PartitionSettings());
val2.calculateHashes();
assertNotNull(val1);
assertEquals(val1, val2);
assertThat("").isNotEqualTo(val1);
validateEquals(val1, val2);
}
@Test
public void equalsIsTrueForOptimizedSearchParam() {
BaseResourceIndexedSearchParamQuantity param = createParam("NAME", "123.001", "value", "VALUE");
BaseResourceIndexedSearchParamQuantity param2 = createParam("NAME", "123.001", "value", "VALUE");
param2.optimizeIndexStorage();
validateEquals(param, param2);
}
private void validateEquals(BaseResourceIndexedSearchParamQuantity theParam1,
BaseResourceIndexedSearchParamQuantity theParam2) {
assertEquals(theParam2, theParam1);
assertEquals(theParam1, theParam2);
assertEquals(theParam1.hashCode(), theParam2.hashCode());
}
@ParameterizedTest
@CsvSource({
"Patient, param, 123.0, units, kg, false, Observation, param, 123.0, units, kg, false, ResourceType is different",
"Patient, param, 123.0, units, kg, false, Patient, name, 123.0, units, kg, false, ParamName is different",
"Patient, param, 123.0, units, kg, false, Patient, param, 321.0, units, kg, false, Value is different",
"Patient, param, 123.0, units, kg, false, Patient, param, 123.0, unitsDiff, kg, false, System is different",
"Patient, param, 123.0, units, kg, false, Patient, param, 123.0, units, lb, false, Units is different",
"Patient, param, 123.0, units, kg, false, Patient, param, 123.0, units, kg, true, Missing is different",
})
public void testEqualsAndHashCode_withDifferentParams_equalsIsFalseAndHashCodeIsDifferent(String theFirstResourceType,
String theFirstParamName,
double theFirstValue,
String theFirstSystem,
String theFirstUnits,
boolean theFirstMissing,
String theSecondResourceType,
String theSecondParamName,
double theSecondValue,
String theSecondSystem,
String theSecondUnits,
boolean theSecondMissing,
String theMessage) {
BaseResourceIndexedSearchParamQuantity param = new ResourceIndexedSearchParamQuantity(
new PartitionSettings(), theFirstResourceType, theFirstParamName, new BigDecimal(theFirstValue), theFirstSystem, theFirstUnits);
param.setMissing(theFirstMissing);
BaseResourceIndexedSearchParamQuantity param2 = new ResourceIndexedSearchParamQuantity(
new PartitionSettings(), theSecondResourceType, theSecondParamName, new BigDecimal(theSecondValue), theSecondSystem, theSecondUnits);
param2.setMissing(theSecondMissing);
assertNotEquals(param, param2, theMessage);
assertNotEquals(param2, param, theMessage);
assertNotEquals(param.hashCode(), param2.hashCode(), theMessage);
}
}

61
hapi-fhir-jpaserver-model/src/test/java/ca/uhn/fhir/jpa/model/entity/ResourceIndexedSearchParamStringTest.java
View File

@ -2,12 +2,12 @@ package ca.uhn.fhir.jpa.model.entity;
import ca.uhn.fhir.jpa.model.config.PartitionSettings;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.CsvSource;
import static org.assertj.core.api.Assertions.assertThat;
import static org.junit.jupiter.api.Assertions.assertAll;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertNotEquals;
import static org.junit.jupiter.api.Assertions.assertNotNull;
@SuppressWarnings("SpellCheckingInspection")
public class ResourceIndexedSearchParamStringTest {
@ -85,10 +85,7 @@ public class ResourceIndexedSearchParamStringTest {
val2.setPartitionSettings(new PartitionSettings());
val2.setStorageSettings(new StorageSettings());
val2.calculateHashes();
assertNotNull(val1);
assertEquals(val1, val2);
assertThat("").isNotEqualTo(val1);
validateEquals(val1, val2);
}
@Test
@ -105,9 +102,55 @@ public class ResourceIndexedSearchParamStringTest {
val2.setPartitionSettings(new PartitionSettings().setIncludePartitionInSearchHashes(true));
val2.setStorageSettings(new StorageSettings());
val2.calculateHashes();
assertNotNull(val1);
assertEquals(val1, val2);
assertThat("").isNotEqualTo(val1);
validateEquals(val1, val2);
}
@Test
public void equalsIsTrueForOptimizedSearchParam() {
ResourceIndexedSearchParamString param = new ResourceIndexedSearchParamString(new PartitionSettings(), new StorageSettings(), "Patient", "param", "aaa", "AAA");
ResourceIndexedSearchParamString param2 = new ResourceIndexedSearchParamString(new PartitionSettings(), new StorageSettings(), "Patient", "param", "aaa", "AAA");
param2.optimizeIndexStorage();
validateEquals(param, param2);
}
private void validateEquals(ResourceIndexedSearchParamString theParam1,
ResourceIndexedSearchParamString theParam2) {
assertEquals(theParam2, theParam1);
assertEquals(theParam1, theParam2);
assertEquals(theParam1.hashCode(), theParam2.hashCode());
}
@ParameterizedTest
@CsvSource({
"Patient, param, aaa, AAA, false, Observation, param, aaa, AAA, false, ResourceType is different",
"Patient, param, aaa, AAA, false, Patient, name, aaa, AAA, false, ParamName is different",
"Patient, param, aaa, AAA, false, Patient, param, bbb, AAA, false, Value is different",
"Patient, param, aaa, AAA, false, Patient, param, aaa, BBB, false, ValueNormalized is different",
"Patient, param, aaa, AAA, false, Patient, param, aaa, AAA, true, Missing is different",
})
public void testEqualsAndHashCode_withDifferentParams_equalsIsFalseAndHashCodeIsDifferent(String theFirstResourceType,
String theFirstParamName,
String theFirstValue,
String theFirstValueNormalized,
boolean theFirstMissing,
String theSecondResourceType,
String theSecondParamName,
String theSecondValue,
String theSecondValueNormalized,
boolean theSecondMissing,
String theMessage) {
ResourceIndexedSearchParamString param = new ResourceIndexedSearchParamString(new PartitionSettings(),
new StorageSettings(), theFirstResourceType, theFirstParamName, theFirstValue, theFirstValueNormalized);
param.setMissing(theFirstMissing);
ResourceIndexedSearchParamString param2 = new ResourceIndexedSearchParamString(new PartitionSettings(),
new StorageSettings(), theSecondResourceType, theSecondParamName, theSecondValue, theSecondValueNormalized);
param2.setMissing(theSecondMissing);
assertNotEquals(param, param2, theMessage);
assertNotEquals(param2, param, theMessage);
assertNotEquals(param.hashCode(), param2.hashCode(), theMessage);
}
}

56
hapi-fhir-jpaserver-model/src/test/java/ca/uhn/fhir/jpa/model/entity/ResourceIndexedSearchParamTokenTest.java
View File

@ -2,10 +2,11 @@ package ca.uhn.fhir.jpa.model.entity;
import ca.uhn.fhir.jpa.model.config.PartitionSettings;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.CsvSource;
import static org.assertj.core.api.Assertions.assertThat;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertNotNull;
import static org.junit.jupiter.api.Assertions.assertNotEquals;
public class ResourceIndexedSearchParamTokenTest {
@ -43,9 +44,54 @@ public class ResourceIndexedSearchParamTokenTest {
.setValue("AAA");
val2.setPartitionSettings(new PartitionSettings());
val2.calculateHashes();
assertNotNull(val1);
assertEquals(val1, val2);
assertThat("").isNotEqualTo(val1);
validateEquals(val1, val2);
}
@Test
public void equalsIsTrueForOptimizedSearchParam() {
ResourceIndexedSearchParamToken param = new ResourceIndexedSearchParamToken(new PartitionSettings(), "Patient", "NAME", "SYSTEM", "VALUE");
ResourceIndexedSearchParamToken param2 = new ResourceIndexedSearchParamToken(new PartitionSettings(), "Patient", "NAME", "SYSTEM", "VALUE");
param2.optimizeIndexStorage();
validateEquals(param, param2);
}
private void validateEquals(ResourceIndexedSearchParamToken theParam1,
ResourceIndexedSearchParamToken theParam2) {
assertEquals(theParam2, theParam1);
assertEquals(theParam1, theParam2);
assertEquals(theParam1.hashCode(), theParam2.hashCode());
}
@ParameterizedTest
@CsvSource({
"Patient, param, system, value, false, Observation, param, system, value, false, ResourceType is different",
"Patient, param, system, value, false, Patient, name, system, value, false, ParamName is different",
"Patient, param, system, value, false, Patient, param, sys, value, false, System is different",
"Patient, param, system, value, false, Patient, param, system, val, false, Value is different",
"Patient, param, system, value, false, Patient, param, system, value, true, Missing is different",
})
public void testEqualsAndHashCode_withDifferentParams_equalsIsFalseAndHashCodeIsDifferent(String theFirstResourceType,
String theFirstParamName,
String theFirstSystem,
String theFirstValue,
boolean theFirstMissing,
String theSecondResourceType,
String theSecondParamName,
String theSecondSystem,
String theSecondValue,
boolean theSecondMissing,
String theMessage) {
ResourceIndexedSearchParamToken param = new ResourceIndexedSearchParamToken(
new PartitionSettings(), theFirstResourceType, theFirstParamName, theFirstSystem, theFirstValue);
param.setMissing(theFirstMissing);
ResourceIndexedSearchParamToken param2 = new ResourceIndexedSearchParamToken(
new PartitionSettings(), theSecondResourceType, theSecondParamName, theSecondSystem, theSecondValue);
param2.setMissing(theSecondMissing);
assertNotEquals(param, param2, theMessage);
assertNotEquals(param2, param, theMessage);
assertNotEquals(param.hashCode(), param2.hashCode(), theMessage);
}
}

53
hapi-fhir-jpaserver-model/src/test/java/ca/uhn/fhir/jpa/model/entity/ResourceIndexedSearchParamUriTest.java
View File

@ -2,10 +2,11 @@ package ca.uhn.fhir.jpa.model.entity;
import ca.uhn.fhir.jpa.model.config.PartitionSettings;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.CsvSource;
import static org.assertj.core.api.Assertions.assertThat;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertNotNull;
import static org.junit.jupiter.api.Assertions.assertNotEquals;
public class ResourceIndexedSearchParamUriTest {
@ -29,10 +30,52 @@ public class ResourceIndexedSearchParamUriTest {
.setUri("http://foo");
val2.setPartitionSettings(new PartitionSettings());
val2.calculateHashes();
assertNotNull(val1);
assertEquals(val1, val2);
assertThat("").isNotEqualTo(val1);
validateEquals(val1, val2);
}
@Test
public void equalsIsTrueForOptimizedSearchParam() {
ResourceIndexedSearchParamUri param = new ResourceIndexedSearchParamUri(new PartitionSettings(), "Patient", "NAME", "http://foo");
ResourceIndexedSearchParamUri param2 = new ResourceIndexedSearchParamUri(new PartitionSettings(), "Patient", "NAME", "http://foo");
param2.optimizeIndexStorage();
validateEquals(param, param2);
}
private void validateEquals(ResourceIndexedSearchParamUri theParam1,
ResourceIndexedSearchParamUri theParam2) {
assertEquals(theParam2, theParam1);
assertEquals(theParam1, theParam2);
assertEquals(theParam1.hashCode(), theParam2.hashCode());
}
@ParameterizedTest
@CsvSource({
"Patient, param, http://test, false, Observation, param, http://test, false, ResourceType is different",
"Patient, param, http://test, false, Patient, name, http://test, false, ParamName is different",
"Patient, param, http://test, false, Patient, param, http://diff, false, Uri is different",
"Patient, param, http://test, false, Patient, param, http://test, true, Missing is different",
})
public void testEqualsAndHashCode_withDifferentParams_equalsIsFalseAndHashCodeIsDifferent(String theFirstResourceType,
String theFirstParamName,
String theFirstUri,
boolean theFirstMissing,
String theSecondResourceType,
String theSecondParamName,
String theSecondUri,
boolean theSecondMissing,
String theMessage) {
ResourceIndexedSearchParamUri param = new ResourceIndexedSearchParamUri(new PartitionSettings(),
theFirstResourceType, theFirstParamName, theFirstUri);
param.setMissing(theFirstMissing);
ResourceIndexedSearchParamUri param2 = new ResourceIndexedSearchParamUri(new PartitionSettings(),
theSecondResourceType, theSecondParamName, theSecondUri);
param2.setMissing(theSecondMissing);
assertNotEquals(param, param2, theMessage);
assertNotEquals(param2, param, theMessage);
assertNotEquals(param.hashCode(), param2.hashCode(), theMessage);
}
}

68
hapi-fhir-jpaserver-model/src/test/java/ca/uhn/fhir/jpa/model/util/SearchParamHashUtilTest.java Normal file
View File

@ -0,0 +1,68 @@
package ca.uhn.fhir.jpa.model.util;
import ca.uhn.fhir.i18n.Msg;
import ca.uhn.fhir.interceptor.model.RequestPartitionId;
import ca.uhn.fhir.jpa.model.config.PartitionSettings;
import ca.uhn.fhir.rest.server.exceptions.InternalErrorException;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.fail;
public class SearchParamHashUtilTest {
private final PartitionSettings myPartitionSettings = new PartitionSettings();
@BeforeEach
void setUp() {
myPartitionSettings.setPartitioningEnabled(false);
}
@Test
public void hashSearchParam_withPartitionDisabled_generatesCorrectHashIdentity() {
Long hashIdentity = SearchParamHash.hashSearchParam(myPartitionSettings, null, "Patient", "name");
// Make sure hashing function gives consistent results
assertEquals(-1575415002568401616L, hashIdentity);
}
@Test
public void hashSearchParam_withPartitionDisabledAndNullValue_generatesCorrectHashIdentity() {
Long hashIdentity = SearchParamHash.hashSearchParam(myPartitionSettings, null, "Patient", "name", null);
// Make sure hashing function gives consistent results
assertEquals(-440750991942222070L, hashIdentity);
}
@Test
public void hashSearchParam_withIncludePartitionInSearchHashesAndNullRequestPartitionId_doesNotThrowException() {
myPartitionSettings.setPartitioningEnabled(true);
myPartitionSettings.setIncludePartitionInSearchHashes(true);
Long hashIdentity = SearchParamHash.hashSearchParam(myPartitionSettings, null, "Patient", "name");
assertEquals(-1575415002568401616L, hashIdentity);
}
@Test
public void hashSearchParam_withIncludePartitionInSearchHashesAndRequestPartitionId_includesPartitionIdInHash() {
myPartitionSettings.setPartitioningEnabled(true);
myPartitionSettings.setIncludePartitionInSearchHashes(true);
RequestPartitionId requestPartitionId = RequestPartitionId.fromPartitionId(1);
Long hashIdentity = SearchParamHash.hashSearchParam(myPartitionSettings, requestPartitionId, "Patient", "name");
assertEquals(-6667609654163557704L, hashIdentity);
}
@Test
public void hashSearchParam_withIncludePartitionInSearchHashesAndMultipleRequestPartitionIds_throwsException() {
myPartitionSettings.setPartitioningEnabled(true);
myPartitionSettings.setIncludePartitionInSearchHashes(true);
RequestPartitionId requestPartitionId = RequestPartitionId.fromPartitionIds(1, 2);
try {
SearchParamHash.hashSearchParam(myPartitionSettings, requestPartitionId, "Patient", "name");
fail();
} catch (InternalErrorException e) {
assertEquals(Msg.code(1527) + "Can not search multiple partitions when partitions are included in search hashes", e.getMessage());
}
}
}

19
hapi-fhir-jpaserver-searchparam/src/main/java/ca/uhn/fhir/jpa/searchparam/extractor/ResourceIndexedSearchParams.java
View File

@ -20,6 +20,7 @@
package ca.uhn.fhir.jpa.searchparam.extractor;
import ca.uhn.fhir.context.RuntimeSearchParam;
import ca.uhn.fhir.interceptor.model.RequestPartitionId;
import ca.uhn.fhir.jpa.model.config.PartitionSettings;
import ca.uhn.fhir.jpa.model.entity.BaseResourceIndexedSearchParam;
import ca.uhn.fhir.jpa.model.entity.NormalizedQuantitySearchLevel;
@ -37,6 +38,7 @@ import ca.uhn.fhir.jpa.model.entity.ResourceLink;
import ca.uhn.fhir.jpa.model.entity.ResourceTable;
import ca.uhn.fhir.jpa.model.entity.SearchParamPresentEntity;
import ca.uhn.fhir.jpa.model.entity.StorageSettings;
import ca.uhn.fhir.jpa.model.util.SearchParamHash;
import ca.uhn.fhir.jpa.model.util.UcumServiceUtil;
import ca.uhn.fhir.jpa.searchparam.util.RuntimeSearchParamHelper;
import ca.uhn.fhir.model.api.IQueryParameterType;
@ -294,7 +296,7 @@ public final class ResourceIndexedSearchParams {
}
for (BaseResourceIndexedSearchParam nextParam : resourceParams) {
if (nextParam.getParamName().equalsIgnoreCase(theParamName)) {
if (isMatchSearchParam(theStorageSettings, theResourceName, theParamName, nextParam)) {
if (nextParam.matches(value)) {
return true;
}
@ -304,6 +306,21 @@ public final class ResourceIndexedSearchParams {
return false;
}
public static boolean isMatchSearchParam(
StorageSettings theStorageSettings,
String theResourceName,
String theParamName,
BaseResourceIndexedSearchParam theIndexedSearchParam) {
if (theStorageSettings.isIndexStorageOptimized()) {
Long hashIdentity = SearchParamHash.hashSearchParam(
new PartitionSettings(), RequestPartitionId.defaultPartition(), theResourceName, theParamName);
return theIndexedSearchParam.getHashIdentity().equals(hashIdentity);
} else {
return theIndexedSearchParam.getParamName().equalsIgnoreCase(theParamName);
}
}
/**
* @deprecated Replace with the method below
*/

5
hapi-fhir-jpaserver-searchparam/src/main/java/ca/uhn/fhir/jpa/searchparam/matcher/InMemoryResourceMatcher.java
View File

@ -71,6 +71,7 @@ import java.util.Map;
import java.util.Set;
import java.util.stream.Collectors;
import static ca.uhn.fhir.jpa.searchparam.extractor.ResourceIndexedSearchParams.isMatchSearchParam;
import static org.apache.commons.lang3.StringUtils.isBlank;
import static org.apache.commons.lang3.StringUtils.isNotBlank;
@ -579,11 +580,11 @@ public class InMemoryResourceMatcher {
switch (theQueryParam.getModifier()) {
case IN:
return theSearchParams.myTokenParams.stream()
.filter(t -> t.getParamName().equals(theParamName))
.filter(t -> isMatchSearchParam(theStorageSettings, theResourceName, theParamName, t))
.anyMatch(t -> systemContainsCode(theQueryParam, t));
case NOT_IN:
return theSearchParams.myTokenParams.stream()
.filter(t -> t.getParamName().equals(theParamName))
.filter(t -> isMatchSearchParam(theStorageSettings, theResourceName, theParamName, t))
.noneMatch(t -> systemContainsCode(theQueryParam, t));
case NOT:
return !theSearchParams.matchParam(

57
hapi-fhir-jpaserver-searchparam/src/main/java/ca/uhn/fhir/jpa/searchparam/registry/JpaSearchParamCache.java
View File

@ -25,9 +25,15 @@ import ca.uhn.fhir.context.phonetic.IPhoneticEncoder;
import ca.uhn.fhir.interceptor.api.HookParams;
import ca.uhn.fhir.interceptor.api.IInterceptorService;
import ca.uhn.fhir.interceptor.api.Pointcut;
import ca.uhn.fhir.interceptor.model.RequestPartitionId;
import ca.uhn.fhir.jpa.model.config.PartitionSettings;
import ca.uhn.fhir.jpa.model.search.StorageProcessingMessage;
import ca.uhn.fhir.jpa.model.util.SearchParamHash;
import ca.uhn.fhir.rest.api.Constants;
import ca.uhn.fhir.rest.api.RestSearchParameterTypeEnum;
import ca.uhn.fhir.rest.api.server.RequestDetails;
import ca.uhn.fhir.rest.server.servlet.ServletRequestDetails;
import ca.uhn.fhir.rest.server.util.IndexedSearchParam;
import ca.uhn.fhir.rest.server.util.ResourceSearchParams;
import org.hl7.fhir.instance.model.api.IIdType;
import org.slf4j.Logger;
@ -46,14 +52,26 @@ import java.util.Set;
import java.util.TreeSet;
import java.util.stream.Collectors;
import static ca.uhn.fhir.rest.api.RestSearchParameterTypeEnum.DATE;
import static ca.uhn.fhir.rest.api.RestSearchParameterTypeEnum.NUMBER;
import static ca.uhn.fhir.rest.api.RestSearchParameterTypeEnum.QUANTITY;
import static ca.uhn.fhir.rest.api.RestSearchParameterTypeEnum.REFERENCE;
import static ca.uhn.fhir.rest.api.RestSearchParameterTypeEnum.SPECIAL;
import static ca.uhn.fhir.rest.api.RestSearchParameterTypeEnum.STRING;
import static ca.uhn.fhir.rest.api.RestSearchParameterTypeEnum.TOKEN;
import static ca.uhn.fhir.rest.api.RestSearchParameterTypeEnum.URI;
import static org.apache.commons.lang3.StringUtils.isNotBlank;
public class JpaSearchParamCache {
private static final Logger ourLog = LoggerFactory.getLogger(JpaSearchParamCache.class);
private static final List<RestSearchParameterTypeEnum> SUPPORTED_INDEXED_SEARCH_PARAMS =
List.of(SPECIAL, DATE, NUMBER, QUANTITY, STRING, TOKEN, URI, REFERENCE);
volatile Map<String, List<RuntimeSearchParam>> myActiveComboSearchParams = Collections.emptyMap();
volatile Map<String, Map<Set<String>, List<RuntimeSearchParam>>> myActiveParamNamesToComboSearchParams =
Collections.emptyMap();
volatile Map<Long, IndexedSearchParam> myHashIdentityToIndexedSearchParams = Collections.emptyMap();
public List<RuntimeSearchParam> getActiveComboSearchParams(String theResourceName) {
List<RuntimeSearchParam> retval = myActiveComboSearchParams.get(theResourceName);
@ -90,6 +108,10 @@ public class JpaSearchParamCache {
return Collections.unmodifiableList(retVal);
}
public Optional<IndexedSearchParam> getIndexedSearchParamByHashIdentity(Long theHashIdentity) {
return Optional.ofNullable(myHashIdentityToIndexedSearchParams.get(theHashIdentity));
}
void populateActiveSearchParams(
IInterceptorService theInterceptorBroadcaster,
IPhoneticEncoder theDefaultPhoneticEncoder,
@ -99,6 +121,7 @@ public class JpaSearchParamCache {
Map<String, RuntimeSearchParam> idToRuntimeSearchParam = new HashMap<>();
List<RuntimeSearchParam> jpaSearchParams = new ArrayList<>();
Map<Long, IndexedSearchParam> hashIdentityToIndexedSearchParams = new HashMap<>();
/*
* Loop through parameters and find JPA params
@ -133,6 +156,7 @@ public class JpaSearchParamCache {
}
setPhoneticEncoder(theDefaultPhoneticEncoder, nextCandidate);
populateIndexedSearchParams(theResourceName, nextCandidate, hashIdentityToIndexedSearchParams);
}
}
@ -183,6 +207,7 @@ public class JpaSearchParamCache {
myActiveComboSearchParams = resourceNameToComboSearchParams;
myActiveParamNamesToComboSearchParams = activeParamNamesToComboSearchParams;
myHashIdentityToIndexedSearchParams = hashIdentityToIndexedSearchParams;
}
void setPhoneticEncoder(IPhoneticEncoder theDefaultPhoneticEncoder, RuntimeSearchParam searchParam) {
@ -195,4 +220,36 @@ public class JpaSearchParamCache {
searchParam.setPhoneticEncoder(theDefaultPhoneticEncoder);
}
}
private void populateIndexedSearchParams(
String theResourceName,
RuntimeSearchParam theRuntimeSearchParam,
Map<Long, IndexedSearchParam> theHashIdentityToIndexedSearchParams) {
if (SUPPORTED_INDEXED_SEARCH_PARAMS.contains(theRuntimeSearchParam.getParamType())) {
addIndexedSearchParam(
theResourceName, theHashIdentityToIndexedSearchParams, theRuntimeSearchParam.getName());
// handle token search parameters with :of-type modifier
if (theRuntimeSearchParam.getParamType() == TOKEN) {
addIndexedSearchParam(
theResourceName,
theHashIdentityToIndexedSearchParams,
theRuntimeSearchParam.getName() + Constants.PARAMQUALIFIER_TOKEN_OF_TYPE);
}
// handle Uplifted Ref Chain Search Parameters
theRuntimeSearchParam.getUpliftRefchainCodes().stream()
.map(urCode -> String.format("%s.%s", theRuntimeSearchParam.getName(), urCode))
.forEach(urSpName ->
addIndexedSearchParam(theResourceName, theHashIdentityToIndexedSearchParams, urSpName));
}
}
private void addIndexedSearchParam(
String theResourceName,
Map<Long, IndexedSearchParam> theHashIdentityToIndexedSearchParams,
String theSpName) {
Long hashIdentity = SearchParamHash.hashSearchParam(
new PartitionSettings(), RequestPartitionId.defaultPartition(), theResourceName, theSpName);
theHashIdentityToIndexedSearchParams.put(hashIdentity, new IndexedSearchParam(theSpName, theResourceName));
}
}

12
hapi-fhir-jpaserver-searchparam/src/main/java/ca/uhn/fhir/jpa/searchparam/registry/SearchParamRegistryImpl.java
View File

@ -31,12 +31,14 @@ import ca.uhn.fhir.jpa.cache.IResourceChangeListenerCache;
import ca.uhn.fhir.jpa.cache.IResourceChangeListenerRegistry;
import ca.uhn.fhir.jpa.cache.ResourceChangeResult;
import ca.uhn.fhir.jpa.model.entity.StorageSettings;
import ca.uhn.fhir.jpa.model.search.ISearchParamHashIdentityRegistry;
import ca.uhn.fhir.jpa.searchparam.SearchParameterMap;
import ca.uhn.fhir.rest.api.Constants;
import ca.uhn.fhir.rest.api.RestSearchParameterTypeEnum;
import ca.uhn.fhir.rest.api.server.IBundleProvider;
import ca.uhn.fhir.rest.server.exceptions.ResourceNotFoundException;
import ca.uhn.fhir.rest.server.util.ISearchParamRegistry;
import ca.uhn.fhir.rest.server.util.IndexedSearchParam;
import ca.uhn.fhir.rest.server.util.ResourceSearchParams;
import ca.uhn.fhir.util.SearchParameterUtil;
import ca.uhn.fhir.util.StopWatch;
@ -65,7 +67,10 @@ import java.util.Set;
import static org.apache.commons.lang3.StringUtils.isBlank;
public class SearchParamRegistryImpl
implements ISearchParamRegistry, IResourceChangeListener, ISearchParamRegistryController {
implements ISearchParamRegistry,
IResourceChangeListener,
ISearchParamRegistryController,
ISearchParamHashIdentityRegistry {
public static final Set<String> NON_DISABLEABLE_SEARCH_PARAMS =
Collections.unmodifiableSet(Sets.newHashSet("*:url", "Subscription:*", "SearchParameter:*"));
@ -147,6 +152,11 @@ public class SearchParamRegistryImpl
return myJpaSearchParamCache.getActiveComboSearchParams(theResourceName, theParamNames);
}
@Override
public Optional<IndexedSearchParam> getIndexedSearchParamByHashIdentity(Long theHashIdentity) {
return myJpaSearchParamCache.getIndexedSearchParamByHashIdentity(theHashIdentity);
}
@Nullable
@Override
public RuntimeSearchParam getActiveSearchParamByUrl(String theUrl) {

34
hapi-fhir-jpaserver-searchparam/src/test/java/ca/uhn/fhir/jpa/searchparam/extractor/ResourceIndexedSearchParamsTest.java
View File

@ -1,5 +1,6 @@
package ca.uhn.fhir.jpa.searchparam.extractor;
import ca.uhn.fhir.jpa.model.entity.ResourceIndexedSearchParamString;
import ca.uhn.fhir.jpa.model.entity.ResourceLink;
import ca.uhn.fhir.jpa.model.entity.ResourceTable;
import ca.uhn.fhir.jpa.model.entity.StorageSettings;
@ -7,6 +8,8 @@ import ca.uhn.fhir.rest.param.ReferenceParam;
import com.google.common.collect.Lists;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.CsvSource;
import java.util.Date;
import java.util.List;
@ -103,4 +106,35 @@ public class ResourceIndexedSearchParamsTest {
assertThat(values).as(values.toString()).isEmpty();
}
@ParameterizedTest
@CsvSource({
"name, name, , false, true",
"name, NAME, , false, true",
"name, name, 7000, false, true",
"name, param, , false, false",
"name, param, 7000, false, false",
" , name, -1575415002568401616, true, true",
"param, name, -1575415002568401616, true, true",
" , param, -1575415002568401616, true, false",
"name, param, -1575415002568401616, true, false",
})
public void testIsMatchSearchParams_matchesByParamNameOrHashIdentity(String theParamName,
String theExpectedParamName,
Long theHashIdentity,
boolean theIndexStorageOptimized,
boolean theShouldMatch) {
// setup
StorageSettings storageSettings = new StorageSettings();
storageSettings.setIndexStorageOptimized(theIndexStorageOptimized);
ResourceIndexedSearchParamString param = new ResourceIndexedSearchParamString();
param.setResourceType("Patient");
param.setParamName(theParamName);
param.setHashIdentity(theHashIdentity);
// execute
boolean isMatch = ResourceIndexedSearchParams.isMatchSearchParam(storageSettings, "Patient", theExpectedParamName, param);
// validate
assertThat(isMatch).isEqualTo(theShouldMatch);
}
}

45
hapi-fhir-jpaserver-searchparam/src/test/java/ca/uhn/fhir/jpa/searchparam/matcher/InMemoryResourceMatcherR5IndexStorageOptimizedTest.java Normal file
View File

@ -0,0 +1,45 @@
package ca.uhn.fhir.jpa.searchparam.matcher;
import ca.uhn.fhir.jpa.model.entity.ResourceIndexedSearchParamDate;
import ca.uhn.fhir.jpa.model.entity.ResourceIndexedSearchParamToken;
import ca.uhn.fhir.jpa.model.entity.ResourceIndexedSearchParamUri;
import org.hl7.fhir.r5.model.Observation;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
public class InMemoryResourceMatcherR5IndexStorageOptimizedTest extends InMemoryResourceMatcherR5Test {
@Override
@BeforeEach
public void before() {
super.before();
myStorageSettings.setIndexStorageOptimized(true);
}
@AfterEach
public void after() {
myStorageSettings.setIndexStorageOptimized(false);
}
@Override
protected ResourceIndexedSearchParamDate extractEffectiveDateParam(Observation theObservation) {
ResourceIndexedSearchParamDate searchParamDate = super.extractEffectiveDateParam(theObservation);
searchParamDate.optimizeIndexStorage();
return searchParamDate;
}
@Override
protected ResourceIndexedSearchParamToken extractCodeTokenParam(Observation theObservation) {
ResourceIndexedSearchParamToken searchParamToken = super.extractCodeTokenParam(theObservation);
searchParamToken.optimizeIndexStorage();
return searchParamToken;
}
@Override
protected ResourceIndexedSearchParamUri extractSourceUriParam(Observation theObservation) {
ResourceIndexedSearchParamUri searchParamUri = super.extractSourceUriParam(theObservation);
searchParamUri.optimizeIndexStorage();
return searchParamUri;
}
}

12
hapi-fhir-jpaserver-searchparam/src/test/java/ca/uhn/fhir/jpa/searchparam/matcher/InMemoryResourceMatcherR5Test.java
View File

@ -34,6 +34,7 @@ import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.mock.mockito.MockBean;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.test.context.ContextConfiguration;
import org.springframework.test.context.junit.jupiter.SpringExtension;
import java.time.Duration;
@ -51,6 +52,7 @@ import static org.mockito.Mockito.verify;
import static org.mockito.Mockito.when;
@ExtendWith(SpringExtension.class)
@ContextConfiguration(classes = {InMemoryResourceMatcherR5Test.SpringConfig.class})
public class InMemoryResourceMatcherR5Test {
public static final String OBSERVATION_DATE = "1970-10-17";
public static final String OBSERVATION_DATETIME = OBSERVATION_DATE + "T01:00:00-08:30";
@ -76,6 +78,8 @@ public class InMemoryResourceMatcherR5Test {
IndexedSearchParamExtractor myIndexedSearchParamExtractor;
@Autowired
private InMemoryResourceMatcher myInMemoryResourceMatcher;
@Autowired
StorageSettings myStorageSettings;
private Observation myObservation;
private ResourceIndexedSearchParams mySearchParams;
@ -414,17 +418,17 @@ public class InMemoryResourceMatcherR5Test {
}
@Nonnull
private ResourceIndexedSearchParamDate extractEffectiveDateParam(Observation theObservation) {
protected ResourceIndexedSearchParamDate extractEffectiveDateParam(Observation theObservation) {
BaseDateTimeType dateValue = (BaseDateTimeType) theObservation.getEffective();
return new ResourceIndexedSearchParamDate(new PartitionSettings(), "Patient", "date", dateValue.getValue(), dateValue.getValueAsString(), dateValue.getValue(), dateValue.getValueAsString(), dateValue.getValueAsString());
return new ResourceIndexedSearchParamDate(new PartitionSettings(), "Observation", "date", dateValue.getValue(), dateValue.getValueAsString(), dateValue.getValue(), dateValue.getValueAsString(), dateValue.getValueAsString());
}
private ResourceIndexedSearchParamToken extractCodeTokenParam(Observation theObservation) {
protected ResourceIndexedSearchParamToken extractCodeTokenParam(Observation theObservation) {
Coding coding = theObservation.getCode().getCodingFirstRep();
return new ResourceIndexedSearchParamToken(new PartitionSettings(), "Observation", "code", coding.getSystem(), coding.getCode());
}
private ResourceIndexedSearchParamUri extractSourceUriParam(Observation theObservation) {
protected ResourceIndexedSearchParamUri extractSourceUriParam(Observation theObservation) {
String source = theObservation.getMeta().getSource();
return new ResourceIndexedSearchParamUri(new PartitionSettings(), "Observation", "_source", source);
}

46
hapi-fhir-jpaserver-searchparam/src/test/java/ca/uhn/fhir/jpa/searchparam/registry/JpaSearchParamCacheTest.java
View File

@ -1,17 +1,25 @@
package ca.uhn.fhir.jpa.searchparam.registry;
import ca.uhn.fhir.context.ComboSearchParamType;
import ca.uhn.fhir.context.FhirContext;
import ca.uhn.fhir.context.RuntimeResourceDefinition;
import ca.uhn.fhir.context.RuntimeSearchParam;
import ca.uhn.fhir.jpa.model.config.PartitionSettings;
import ca.uhn.fhir.jpa.model.util.SearchParamHash;
import ca.uhn.fhir.rest.server.util.IndexedSearchParam;
import org.hl7.fhir.instance.model.api.IIdType;
import org.hl7.fhir.r4.model.IdType;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.CsvSource;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Optional;
import static ca.uhn.fhir.util.HapiExtensions.EXTENSION_SEARCHPARAM_UPLIFT_REFCHAIN;
import static org.assertj.core.api.Assertions.assertThat;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertTrue;
@ -19,10 +27,11 @@ import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.when;
public class JpaSearchParamCacheTest {
private static final FhirContext ourFhirContext = FhirContext.forR4Cached();
private static final String RESOURCE_TYPE = "Patient";
private TestableJpaSearchParamCache myJpaSearchParamCache;
@BeforeEach
public void beforeEach(){
myJpaSearchParamCache = new TestableJpaSearchParamCache();
@ -93,6 +102,41 @@ public class JpaSearchParamCacheTest {
assertTrue(found.isEmpty());
}
@ParameterizedTest
@CsvSource({
"Patient, name, name, type = string",
"Patient, active, active, type = token",
"Patient, active, active:of-type, type = token with of-type",
"Patient, birthdate, birthdate, type = date",
"Patient, general-practitioner, general-practitioner, type = reference",
"Location, near, near, type = special",
"RiskAssessment, probability, probability, type = number",
"Observation, value-quantity, value-quantity, type = quantity",
"ValueSet, url, url, type = uri",
"Encounter, subject, subject.name, type = reference with refChain"
})
public void getIndexedSearchParamByHashIdentity_returnsCorrectIndexedSearchParam(String theResourceType,
String theSpName,
String theExpectedSpName,
String theSpType) {
// setup
RuntimeSearchParamCache runtimeCache = new RuntimeSearchParamCache();
RuntimeResourceDefinition resourceDefinition = ourFhirContext.getResourceDefinition(theResourceType);
RuntimeSearchParam runtimeSearchParam = resourceDefinition.getSearchParam(theSpName);
runtimeSearchParam.addUpliftRefchain("name", EXTENSION_SEARCHPARAM_UPLIFT_REFCHAIN);
runtimeCache.add(theResourceType, theSpName, resourceDefinition.getSearchParam(theSpName));
Long hashIdentity = SearchParamHash.hashSearchParam(new PartitionSettings(), null, theResourceType, theExpectedSpName);
// execute
myJpaSearchParamCache.populateActiveSearchParams(null, null, runtimeCache);
Optional<IndexedSearchParam> indexedSearchParam = myJpaSearchParamCache.getIndexedSearchParamByHashIdentity(hashIdentity);
// validate
assertTrue(indexedSearchParam.isPresent(), "No IndexedSearchParam found for search param with " + theSpType);
assertEquals(theResourceType, indexedSearchParam.get().getResourceType());
assertEquals(theExpectedSpName, indexedSearchParam.get().getParameterName());
}
private RuntimeSearchParam createSearchParam(ComboSearchParamType theType){
return createSearchParam(null, theType);
}

362
hapi-fhir-jpaserver-test-r4/src/test/java/ca/uhn/fhir/jpa/dao/r4/FhirResourceDaoR4IndexStorageOptimizedTest.java Normal file
View File

@ -0,0 +1,362 @@
package ca.uhn.fhir.jpa.dao.r4;
import ca.uhn.fhir.batch2.api.IJobCoordinator;
import ca.uhn.fhir.batch2.jobs.reindex.ReindexAppCtx;
import ca.uhn.fhir.batch2.jobs.reindex.ReindexJobParameters;
import ca.uhn.fhir.batch2.model.JobInstanceStartRequest;
import ca.uhn.fhir.context.ConfigurationException;
import ca.uhn.fhir.i18n.Msg;
import ca.uhn.fhir.jpa.api.dao.IFhirResourceDao;
import ca.uhn.fhir.jpa.batch.models.Batch2JobStartResponse;
import ca.uhn.fhir.jpa.config.SearchConfig;
import ca.uhn.fhir.jpa.model.config.PartitionSettings;
import ca.uhn.fhir.jpa.model.entity.BaseResourceIndexedSearchParam;
import ca.uhn.fhir.jpa.model.entity.NormalizedQuantitySearchLevel;
import ca.uhn.fhir.jpa.model.entity.ResourceIndexedSearchParamCoords;
import ca.uhn.fhir.jpa.model.entity.ResourceIndexedSearchParamDate;
import ca.uhn.fhir.jpa.model.entity.ResourceIndexedSearchParamNumber;
import ca.uhn.fhir.jpa.model.entity.ResourceIndexedSearchParamQuantity;
import ca.uhn.fhir.jpa.model.entity.ResourceIndexedSearchParamQuantityNormalized;
import ca.uhn.fhir.jpa.model.entity.ResourceIndexedSearchParamString;
import ca.uhn.fhir.jpa.model.entity.ResourceIndexedSearchParamToken;
import ca.uhn.fhir.jpa.model.entity.ResourceIndexedSearchParamUri;
import ca.uhn.fhir.jpa.model.entity.StorageSettings;
import ca.uhn.fhir.jpa.model.util.SearchParamHash;
import ca.uhn.fhir.jpa.model.util.UcumServiceUtil;
import ca.uhn.fhir.jpa.reindex.ReindexStepTest;
import ca.uhn.fhir.jpa.searchparam.SearchParameterMap;
import ca.uhn.fhir.jpa.test.BaseJpaR4Test;
import ca.uhn.fhir.rest.param.BaseParam;
import ca.uhn.fhir.rest.param.DateParam;
import ca.uhn.fhir.rest.param.NumberParam;
import ca.uhn.fhir.rest.param.QuantityParam;
import ca.uhn.fhir.rest.param.SpecialParam;
import ca.uhn.fhir.rest.param.StringParam;
import ca.uhn.fhir.rest.param.TokenParam;
import ca.uhn.fhir.rest.param.UriParam;
import org.hl7.fhir.instance.model.api.IBaseResource;
import org.hl7.fhir.instance.model.api.IIdType;
import org.hl7.fhir.r4.model.DateType;
import org.hl7.fhir.r4.model.DecimalType;
import org.hl7.fhir.r4.model.Location;
import org.hl7.fhir.r4.model.Observation;
import org.hl7.fhir.r4.model.Patient;
import org.hl7.fhir.r4.model.Quantity;
import org.hl7.fhir.r4.model.RiskAssessment;
import org.hl7.fhir.r4.model.Substance;
import org.hl7.fhir.r4.model.ValueSet;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Nested;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.CsvSource;
import org.junit.jupiter.params.provider.ValueSource;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.data.jpa.repository.JpaRepository;
import java.util.List;
import static org.junit.jupiter.api.Assertions.assertDoesNotThrow;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertNotNull;
import static org.junit.jupiter.api.Assertions.assertNull;
import static org.junit.jupiter.api.Assertions.assertTrue;
import static org.junit.jupiter.api.Assertions.fail;
/**
* This test was added to check if changing {@link StorageSettings#isIndexStorageOptimized()} setting and performing
* $reindex operation will correctly null/recover sp_name, res_type, sp_updated parameters
* of ResourceIndexedSearchParam entities.
*/
public class FhirResourceDaoR4IndexStorageOptimizedTest extends BaseJpaR4Test {
@Autowired
private IJobCoordinator myJobCoordinator;
@Autowired
private SearchConfig mySearchConfig;
@AfterEach
void cleanUp() {
myPartitionSettings.setIncludePartitionInSearchHashes(false);
}
@ParameterizedTest
@ValueSource(booleans = {true, false})
public void testCoordinatesIndexedSearchParam_searchAndReindex_searchParamUpdatedCorrectly(boolean theIsIndexStorageOptimized) {
// setup
myStorageSettings.setIndexStorageOptimized(theIsIndexStorageOptimized);
Location loc = new Location();
loc.getPosition().setLatitude(43.7);
loc.getPosition().setLongitude(79.4);
IIdType id = myLocationDao.create(loc, mySrd).getId().toUnqualifiedVersionless();
validateAndReindex(theIsIndexStorageOptimized, myLocationDao, myResourceIndexedSearchParamCoordsDao, id,
Location.SP_NEAR, "Location", new SpecialParam().setValue("43.7|79.4"), ResourceIndexedSearchParamCoords.class);
}
@ParameterizedTest
@ValueSource(booleans = {true, false})
public void testDateIndexedSearchParam_searchAndReindex_searchParamUpdatedCorrectly(boolean theIsIndexStorageOptimized) {
// setup
myStorageSettings.setIndexStorageOptimized(theIsIndexStorageOptimized);
Patient p = new Patient();
p.setBirthDateElement(new DateType("2021-02-22"));
IIdType id = myPatientDao.create(p, mySrd).getId().toUnqualifiedVersionless();
validateAndReindex(theIsIndexStorageOptimized, myPatientDao, myResourceIndexedSearchParamDateDao, id,
Patient.SP_BIRTHDATE, "Patient", new DateParam("2021-02-22"), ResourceIndexedSearchParamDate.class);
}
@ParameterizedTest
@ValueSource(booleans = {true, false})
public void testNumberIndexedSearchParam_searchAndReindex_searchParamUpdatedCorrectly(boolean theIsIndexStorageOptimized) {
// setup
myStorageSettings.setIndexStorageOptimized(theIsIndexStorageOptimized);
RiskAssessment riskAssessment = new RiskAssessment();
DecimalType doseNumber = new DecimalType(15);
riskAssessment.addPrediction(new RiskAssessment.RiskAssessmentPredictionComponent().setProbability(doseNumber));
IIdType id = myRiskAssessmentDao.create(riskAssessment, mySrd).getId().toUnqualifiedVersionless();
validateAndReindex(theIsIndexStorageOptimized, myRiskAssessmentDao, myResourceIndexedSearchParamNumberDao, id,
RiskAssessment.SP_PROBABILITY, "RiskAssessment", new NumberParam(15), ResourceIndexedSearchParamNumber.class);
}
@ParameterizedTest
@ValueSource(booleans = {true, false})
public void testQuantityIndexedSearchParam_searchAndReindex_searchParamUpdatedCorrectly(boolean theIsIndexStorageOptimized) {
// setup
myStorageSettings.setIndexStorageOptimized(theIsIndexStorageOptimized);
Observation observation = new Observation();
observation.setValue(new Quantity(123));
IIdType id = myObservationDao.create(observation, mySrd).getId().toUnqualifiedVersionless();
validateAndReindex(theIsIndexStorageOptimized, myObservationDao, myResourceIndexedSearchParamQuantityDao, id,
Observation.SP_VALUE_QUANTITY, "Observation", new QuantityParam(123), ResourceIndexedSearchParamQuantity.class);
}
@ParameterizedTest
@ValueSource(booleans = {true, false})
public void testQuantityNormalizedIndexedSearchParam_searchAndReindex_searchParamUpdatedCorrectly(boolean theIsIndexStorageOptimized) {
// setup
myStorageSettings.setIndexStorageOptimized(theIsIndexStorageOptimized);
myStorageSettings.setNormalizedQuantitySearchLevel(NormalizedQuantitySearchLevel.NORMALIZED_QUANTITY_STORAGE_SUPPORTED);
Substance res = new Substance();
res.addInstance().getQuantity().setSystem(UcumServiceUtil.UCUM_CODESYSTEM_URL).setCode("m").setValue(123);
IIdType id = mySubstanceDao.create(res, mySrd).getId().toUnqualifiedVersionless();
QuantityParam quantityParam = new QuantityParam(null, 123, UcumServiceUtil.UCUM_CODESYSTEM_URL, "m");
validateAndReindex(theIsIndexStorageOptimized, mySubstanceDao, myResourceIndexedSearchParamQuantityNormalizedDao,
id, Substance.SP_QUANTITY, "Substance", quantityParam, ResourceIndexedSearchParamQuantityNormalized.class);
}
@ParameterizedTest
@ValueSource(booleans = {true, false})
public void testStringIndexedSearchParam_searchAndReindex_searchParamUpdatedCorrectly(boolean theIsIndexStorageOptimized) {
// setup
myStorageSettings.setIndexStorageOptimized(theIsIndexStorageOptimized);
Patient p = new Patient();
p.addAddress().addLine("123 Main Street");
IIdType id = myPatientDao.create(p, mySrd).getId().toUnqualifiedVersionless();
validateAndReindex(theIsIndexStorageOptimized, myPatientDao, myResourceIndexedSearchParamStringDao, id,
Patient.SP_ADDRESS, "Patient", new StringParam("123 Main Street"), ResourceIndexedSearchParamString.class);
}
@ParameterizedTest
@ValueSource(booleans = {true, false})
public void testTokenIndexedSearchParam_searchAndReindex_searchParamUpdatedCorrectly(boolean theIsIndexStorageOptimized) {
// setup
myStorageSettings.setIndexStorageOptimized(theIsIndexStorageOptimized);
Observation observation = new Observation();
observation.setStatus(Observation.ObservationStatus.FINAL);
IIdType id = myObservationDao.create(observation, mySrd).getId().toUnqualifiedVersionless();
validateAndReindex(theIsIndexStorageOptimized, myObservationDao, myResourceIndexedSearchParamTokenDao, id,
Observation.SP_STATUS, "Observation", new TokenParam("final"), ResourceIndexedSearchParamToken.class);
}
@ParameterizedTest
@ValueSource(booleans = {true, false})
public void testUriIndexedSearchParam_searchAndReindex_searchParamUpdatedCorrectly(boolean theIsIndexStorageOptimized) {
// setup
myStorageSettings.setIndexStorageOptimized(theIsIndexStorageOptimized);
ValueSet valueSet = new ValueSet();
valueSet.setUrl("http://vs");
IIdType id = myValueSetDao.create(valueSet, mySrd).getId().toUnqualifiedVersionless();
validateAndReindex(theIsIndexStorageOptimized, myValueSetDao, myResourceIndexedSearchParamUriDao, id,
ValueSet.SP_URL, "ValueSet", new UriParam("http://vs"), ResourceIndexedSearchParamUri.class);
}
@ParameterizedTest
@CsvSource({
"false, false, false",
"false, false, true",
"false, true, false",
"true, false, false",
"true, false, true",
"true, true, false"})
public void testValidateConfiguration_withCorrectConfiguration_doesNotThrowException(boolean thePartitioningEnabled,
boolean theIsIncludePartitionInSearchHashes,
boolean theIsIndexStorageOptimized) {
myPartitionSettings.setPartitioningEnabled(thePartitioningEnabled);
myPartitionSettings.setIncludePartitionInSearchHashes(theIsIncludePartitionInSearchHashes);
myStorageSettings.setIndexStorageOptimized(theIsIndexStorageOptimized);
assertDoesNotThrow(() -> mySearchConfig.validateConfiguration());
}
@Test
public void testValidateConfiguration_withInCorrectConfiguration_throwsException() {
myPartitionSettings.setIncludePartitionInSearchHashes(true);
myPartitionSettings.setPartitioningEnabled(true);
myStorageSettings.setIndexStorageOptimized(true);
try {
mySearchConfig.validateConfiguration();
fail();
} catch (ConfigurationException e) {
assertEquals(Msg.code(2525) + "Incorrect configuration. "
+ "StorageSettings#isIndexStorageOptimized and PartitionSettings.isIncludePartitionInSearchHashes "
+ "cannot be enabled at the same time.", e.getMessage());
}
}
private void validateAndReindex(boolean theIsIndexStorageOptimized, IFhirResourceDao<? extends IBaseResource> theResourceDao,
JpaRepository<? extends BaseResourceIndexedSearchParam, Long> theIndexedSpRepository, IIdType theId,
String theSearchParam, String theResourceType, BaseParam theParamValue,
Class<? extends BaseResourceIndexedSearchParam> theIndexedSearchParamClass) {
// validate
validateSearchContainsResource(theResourceDao, theId, theSearchParam, theParamValue);
validateSearchParams(theIndexedSpRepository, theId, theSearchParam, theResourceType, theIndexedSearchParamClass);
// switch on/off storage optimization and run $reindex
myStorageSettings.setIndexStorageOptimized(!theIsIndexStorageOptimized);
executeReindex(theResourceType + "?");
// validate again
validateSearchContainsResource(theResourceDao, theId, theSearchParam, theParamValue);
validateSearchParams(theIndexedSpRepository, theId, theSearchParam, theResourceType, theIndexedSearchParamClass);
}
private void validateSearchParams(JpaRepository<? extends BaseResourceIndexedSearchParam, Long> theIndexedSpRepository,
IIdType theId, String theSearchParam, String theResourceType,
Class<? extends BaseResourceIndexedSearchParam> theIndexedSearchParamClass) {
List<? extends BaseResourceIndexedSearchParam> repositorySearchParams =
getAndValidateIndexedSearchParamsRepository(theIndexedSpRepository, theId, theSearchParam, theResourceType);
long hash = SearchParamHash.hashSearchParam(new PartitionSettings(), null, theResourceType, theSearchParam);
if (myStorageSettings.isIndexStorageOptimized()) {
// validated sp_name, res_type, sp_updated columns are null in DB
runInTransaction(() -> {
List<?> results = myEntityManager.createQuery("SELECT i FROM " + theIndexedSearchParamClass.getSimpleName() +
" i WHERE i.myResourcePid = " + theId.getIdPartAsLong() + " AND i.myResourceType IS NULL " +
"AND i.myParamName IS NULL AND i.myUpdated IS NULL AND i.myHashIdentity = " + hash, theIndexedSearchParamClass).getResultList();
assertFalse(results.isEmpty());
assertEquals(repositorySearchParams.size(), results.size());
});
} else {
// validated sp_name, res_type, sp_updated columns are not null in DB
runInTransaction(() -> {
List<?> results = myEntityManager.createQuery("SELECT i FROM " + theIndexedSearchParamClass.getSimpleName() +
" i WHERE i.myResourcePid = " + theId.getIdPartAsLong() + " AND i.myResourceType = '" + theResourceType +
"' AND i.myParamName = '" + theSearchParam + "' AND i.myUpdated IS NOT NULL AND i.myHashIdentity = " + hash,
theIndexedSearchParamClass).getResultList();
assertFalse(results.isEmpty());
assertEquals(repositorySearchParams.size(), results.size());
});
}
}
private List<? extends BaseResourceIndexedSearchParam> getAndValidateIndexedSearchParamsRepository(
JpaRepository<? extends BaseResourceIndexedSearchParam, Long> theIndexedSpRepository,
IIdType theId, String theSearchParam, String theResourceType) {
List<? extends BaseResourceIndexedSearchParam> repositorySearchParams = theIndexedSpRepository.findAll()
.stream()
.filter(sp -> sp.getResourcePid().equals(theId.getIdPartAsLong()))
.filter(sp -> theSearchParam.equals(sp.getParamName()))
.toList();
assertFalse(repositorySearchParams.isEmpty());
repositorySearchParams.forEach(sp -> {
assertEquals(theResourceType, sp.getResourceType());
if (myStorageSettings.isIndexStorageOptimized()) {
assertNull(sp.getUpdated());
} else {
assertNotNull(sp.getUpdated());
}
});
return repositorySearchParams;
}
private void validateSearchContainsResource(IFhirResourceDao<? extends IBaseResource> theResourceDao,
IIdType theId,
String theSearchParam,
BaseParam theParamValue) {
SearchParameterMap searchParameterMap = new SearchParameterMap()
.setLoadSynchronous(true)
.add(theSearchParam, theParamValue);
List<IIdType> listIds = toUnqualifiedVersionlessIds(theResourceDao.search(searchParameterMap));
assertTrue(listIds.contains(theId));
}
private void executeReindex(String... theUrls) {
ReindexJobParameters parameters = new ReindexJobParameters();
for (String url : theUrls) {
parameters.addUrl(url);
}
JobInstanceStartRequest startRequest = new JobInstanceStartRequest();
startRequest.setJobDefinitionId(ReindexAppCtx.JOB_REINDEX);
startRequest.setParameters(parameters);
Batch2JobStartResponse res = myJobCoordinator.startInstance(mySrd, startRequest);
ourLog.info("Started reindex job with id {}", res.getInstanceId());
myBatch2JobHelper.awaitJobCompletion(res);
}
// Additional existing tests with enabled IndexStorageOptimized
@Nested
public class IndexStorageOptimizedReindexStepTest extends ReindexStepTest {
@BeforeEach
void setUp() {
myStorageSettings.setIndexStorageOptimized(true);
}
}
@Nested
public class IndexStorageOptimizedPartitioningSqlR4Test extends PartitioningSqlR4Test {
@BeforeEach
void setUp() {
myStorageSettings.setIndexStorageOptimized(true);
}
}
@Nested
public class IndexStorageOptimizedFhirResourceDaoR4SearchMissingTest extends FhirResourceDaoR4SearchMissingTest {
@BeforeEach
void setUp() {
myStorageSettings.setIndexStorageOptimized(true);
}
}
@Nested
public class IndexStorageOptimizedFhirResourceDaoR4QueryCountTest extends FhirResourceDaoR4QueryCountTest {
@BeforeEach
void setUp() {
myStorageSettings.setIndexStorageOptimized(true);
}
}
@Nested
public class IndexStorageOptimizedFhirResourceDaoR4SearchNoFtTest extends FhirResourceDaoR4SearchNoFtTest {
@BeforeEach
void setUp() {
myStorageSettings.setIndexStorageOptimized(true);
}
}
}

14
hapi-fhir-jpaserver-test-r4/src/test/java/ca/uhn/fhir/jpa/provider/r4/ResourceProviderR4Test.java
View File

@ -7155,6 +7155,20 @@ public class ResourceProviderR4Test extends BaseResourceProviderR4Test {
return new InstantDt(theDate).getValueAsString();
}
@Nested
public class IndexStorageOptimizedMissingSearchParameterTests extends MissingSearchParameterTests {
@BeforeEach
public void init() {
super.init();
myStorageSettings.setIndexStorageOptimized(true);
}
@AfterEach
public void cleanUp() {
myStorageSettings.setIndexStorageOptimized(false);
}
}
@Nested
public class MissingSearchParameterTests {

52
hapi-fhir-jpaserver-test-r5/src/test/java/ca/uhn/fhir/jpa/dao/r5/FhirResourceDaoR5IndexStorageOptimizedTest.java Normal file
View File

@ -0,0 +1,52 @@
package ca.uhn.fhir.jpa.dao.r5;
import ca.uhn.fhir.jpa.model.entity.StorageSettings;
import ca.uhn.fhir.jpa.search.reindex.InstanceReindexServiceImplR5Test;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Nested;
/**
* R5 Test cases with enabled {@link StorageSettings#isIndexStorageOptimized()}
*/
public class FhirResourceDaoR5IndexStorageOptimizedTest {
@Nested
public class IndexStorageOptimizedFhirSystemDaoTransactionR5Test extends FhirSystemDaoTransactionR5Test {
@BeforeEach
public void setUp() {
myStorageSettings.setIndexStorageOptimized(true);
}
@AfterEach
public void cleanUp() {
myStorageSettings.setIndexStorageOptimized(false);
}
}
@Nested
public class IndexStorageOptimizedInstanceReindexServiceImplR5Test extends InstanceReindexServiceImplR5Test {
@BeforeEach
public void setUp() {
myStorageSettings.setIndexStorageOptimized(true);
}
@AfterEach
public void cleanUp() {
myStorageSettings.setIndexStorageOptimized(false);
}
}
@Nested
public class IndexStorageOptimizedUpliftedRefchainsAndChainedSortingR5Test extends UpliftedRefchainsAndChainedSortingR5Test {
@BeforeEach
public void setUp() {
myStorageSettings.setIndexStorageOptimized(true);
}
@AfterEach
public void cleanUp() {
myStorageSettings.setIndexStorageOptimized(false);
}
}
}

42
hapi-fhir-server/src/main/java/ca/uhn/fhir/rest/server/util/IndexedSearchParam.java Normal file
View File

@ -0,0 +1,42 @@
/*-
* #%L
* HAPI FHIR - Server Framework
* %%
* Copyright (C) 2014 - 2024 Smile CDR, Inc.
* %%
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
* #L%
*/
package ca.uhn.fhir.rest.server.util;
/**
* Simplified model of indexed search parameter
*/
public class IndexedSearchParam {
private final String myParameterName;
private final String myResourceType;
public IndexedSearchParam(String theParameterName, String theResourceType) {
this.myParameterName = theParameterName;
this.myResourceType = theResourceType;
}
public String getParameterName() {
return myParameterName;
}
public String getResourceType() {
return myResourceType;
}
}