Add support for _total, _count, and _offset to Lucene backend.
Co-authored-by: Michael Buckley <michael.buckley@smilecdr.com>
Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
* committed changes discussed at tasking
* Changed the subscription message to store the request partition
* modified mdm subscription loader to create a cross partition subscription in the default partition if multitenancy is enabled
* Fix errors when running Multitenant IT for mdm
* Added partition id to mdm golden resource search and creating/updating golden resource
* added partition id column to the mdm link table
* Added partition id of the source resource to partition id column of the mdm link table
* broken build
* working -links, missing hapi tests
* Fixed the mdm link db table due to feedback
* added IT for create link mdm operation, also added changes to pass the IT
* Fixed test for create mdm links, added test for update mdm link on partitioned server
* changed mdmLink search from searchbyExample to criteriaBuilder, added partitionHelperSvc to determine partition
* added test for merge golden resource and search golden resource on partitioned server
* added unit test for not duplicate on partitioned and non-partitioned server
* added criteriabuilder search and tests
* code cleanup
* added partition support to -merge-golden-resources operation
* test cleanup
* fixed operations with multitenancy
* added test for mdm clear operation
* added waiting for batch job to finish in mdm clear operation test
* added query to dao to support partitioned server for mdm clear operation
* resolve mistake from merge conflict
* added mdm-submit operations, also fixed paging for -query
* added partitionIds constant
* fixed build issue where there was no Code.msg in mdm partition exceptions
* code review fixes
* fix broken test, also fix a bug where query link could not query by linkSource properly
* fix test stubbing
* bump hapi version to 6.0.0-PRE10-SNAPSHOT
Co-authored-by: Ken Stevens <khstevens@gmail.com>
Co-authored-by: Steven Li <steven@smilecdr.com>
Co-authored-by: Long Ma <long@smilecdr.com>
* moved classes
* Bump version
* licensing
* Repackage, fix messages
* reject empty streams
* Remove dead import
* Version bump
* Fix up test to actually return a data stream
* Move null binary svc to storage
* Start working on reindex job
* Updates
* Ongoing work
* Ongoing work
* Work on reindex
* Reindex work
* Add logging
* Fix typo
* Test fix
* Test fix
* Address fixme
* Liocense header
* Resolve fixme
* Add logging
* Update logs
* Address review comments
* License header
* Test fixes
* Test fix
* Test fixes
* Test fix
* Version bump
* Version bump
* Rely on context refresh event instead of postconstruct. Load bean lazily.
* Remove parent autowire
* Break termcodesystem circular dependency with new bean
* Break cycle between BaseTermReadSvcImpl and TermCodeSystemStorageSvcImpl
* move back into helper method
* Rip out scheduling and submission to spring batch away from BulkDataExportSvcImpl
* wip
* Fix bean naming
* Revert
* re-add autowiring
* Reformat
* docs and reformat
* Back out change
* feedback from review
* merge conflicts
* Enable search narrowing on large ValueSets
* ValueSet improvements
* Work on narrowing
* Work on narrowing
* Work on narrowing
* Work on narrowing
* Add test
* Work on narrowing interceptor
* Work on narrowing
* License headers
* Refactor code narrowing
* Add docs
* Version bump
* Test fix
* Test fixes
* Build fix
* Fixes
* Version bump
* Test fix
* License header updates
* Build fix
* Test fixes
* Test fix
* Test fix
* Docs fix
* Test fix
* Test fix
* Resolve fixme
* Bump
* Force a CI build
* Make CI happen again
* Work on new batch framework
* Work on new batch framework
* Work on new batch framework
* Work on batch
* Compile working
* Work on bulk import
* Adjust import
* Work
* Work on batch
* Bump version
* WOrk on new batch processes
* Work on bath
* Bump to PRE4
* Build fixes
* CLeanup
* Small tweak
* Add exception code
* Test fixes
* Test fixes
* Test fix
* Additional synchronization
* Add license headers
* Test fixes
* Test fixes
* Add changelogs
* Address PG
* Test fix
* Test fix
* Test fixes
* Review notes
* Work on tests
* Test fixes
* Test fix
* Work on tests
* Tets fix
* Test fixes
* Test fixes
* Add missing exception codes
* Test fix
* Test fixes
* More test fixing
* License headers
* Test fix
* Add new test logging
* Work on tests
* Test fixes
* Test fix
* Resolve fixme
* Try to avoid test failure
* Add import command
* Work on storage
* Fix error codes
* Fixes
* License header
* Build fix
* Build fixes
* Fix dep
* fix handling of common search parameters
* Revert "fix handling of common search parameters"
This reverts commit 89c45eebdc.
* Fix implementation, add test (#3378)
* Fix implementation, add test
* Tighten test
* Rip out dead modules
* Add changelog
Co-authored-by: Jason Roberts <jason.roberts@smilecdr.com>
* Fixed null pointer exception for re-loading subscription on cdr restart and when there's no partition id in the request, and added tests
* added changelogs for this fix
Co-authored-by: Steven Li <steven@smilecdr.com>
* Change token and reference search params to 'keyword' field type
* remove normalizer for keyword field
* WIP - refactor lastN operation to use hibernate search generated ES index.
* Add ES native aggregation builder for lastN
* added date search param support for searching resourcetable ES index
* * Added DateSearch parameterized tests to ElasticSearch test suite
* Added more tests cases for Date searches
* WIP refactor LastN test data to central generator class and use it for both hibernate-search and elastic search implementations.
* WIP fix lastNDataGenerator to avoid assertion errors.
* Fixed tests to run test suite on both existing lastN search implementation and newer extended lucene index based search.
* Cleanup json string to avoid carriage return noise
* Fix checkstyle issues
* Fix checkstyle Msg code issues
* Fix checkstyle Msg code issues
* Add more test case for GT, SA precision modifier
* Fix date precision test cases
* Fix Msg Code conflicts
* Add more logging
* Add way more loggging
* even more logging
* fix msg code conflicts
Co-authored-by: Tadgh <garygrantgraham@gmail.com>
* Initial design of adding exception codes to exceptions
* Initial design of adding exception codes to exceptions
* change int module name to string
* add comment
* shorten message
* shorten message
* added checkstyle
* developing checker
* completed checkstyle plugin and unit test for it
* fix NPE
* fix checker so it ignores throws on exceptions that aren't new
* change parameter order in InvalidResponseException
* accept Msg.code() in any parameter since some exceptions like MessageException have the message string as the second parameter
* exclude generated files
* BIG COMMIT added error codes to all exception throws
* example test fix
* Modified tests such that the assert statements would better match the message responses.
* Modified additional tests such that the assert statements would better match the message responses.
* fix test
* merge master
* merge master
* Fourth commit of modified tests such that the assert statements would better match the message response
* Changes made to modify assert statements such that they match the new errors which now use error codes. Also changed ResourceNotFoundException to include error codes in its messages.
* Modified assert messages to better match new error codes
* Time Zone changes DRAFT. Certain tests pass in intellij and fail during mvn install
* Modified assert messages to better match new error codes
* Added comments for time zone fixes. Previous commit comment on time zone fixes was incorrect. By invaliding caches, all tests on both intellij and mvn install passed.
* Reverted changes on azure-pipelines.yml
* updated hapi-fhir version for checkstyle module
* changes to azure-pipelines.yml
* changes to azure-pipelines.yml
* change to pom.xml to include hapi-fhir-checkstyle in ALLMODULES
* changes to azure-piplines (these changes are to be reverted after testing)
* reverted changes to azure-piplines
* reverted change where hapi-fhir-checkstyle was added to pom.xml (hapi-fhir)
* Attempted fix on checkstyle build error
* modified azure-piplines.yml to change maven repository for checkstyle build
* Attempted fix on checkstyle build error
* checkstyle config location change for hapi-depoloyable-pom. (will be reverted if it does not work)
* attempted dependency add for hapi-depoloyable-pom. (will be reverted if it does not work)
* reverted changes for hapi-deployable-pom
* add javadoc
* update last code
* Attempted fix on checkstyle build error
* Finished adding in missing error codes to pass Checkstyle checks
* Test pipelining
* Fixing errors and bumping version (as suggested by Gary)
* Merge
* Fixing Checkstyle error
* Attempted fix for date error
* Add error code onto assert statement
* Add in a couple more error codes
* Fixing assert statements
* Bump to static version for dependency
* Adding error code to message
* Adding error codes to exceptions
* Adding error codes to exceptions
* Change to work with cdr branch
* Update to faulty error message
* Merge changes
* allow checkstyle pom to deploy
Co-authored-by: Ken Stevens <khstevens@gmail.com>
Co-authored-by: Simon Zuccherato <simon.zuccherato@smilecdr.com>
Co-authored-by: Predap <simon.zuccherato@gmail.com>
Co-authored-by: Predap <46201929+Predap@users.noreply.github.com>
Co-authored-by: Tadgh <garygrantgraham@gmail.com>
* Bump a number of versions
* Fixes
* Work on version bump
* Bump
* Test fix
* More version bumps
* Build fixes
* Test fixes
* Fixes
* Fixes
* Test fix
* Bump jacoco
* Test fix
* Test fix
* Test fix
* Build fix
* Build fix
* Remove versioned H2
* Build fix
* Avoid memory issue in validation tests
* JDK bump
* Restore compile with errorprone
* Force coloured output
* Force jansi
* Try again to force jansi
* Fix for hardcoded date that just passed
* Fixes to JPA
* Improve changelog
* Work on failing test
* Test fix
* Compile fix
* One more version bump
* Test fixes
* some fixmes to start
* Added some FIXMEs
* added RequestPartitionId to reosurceDelivryMessage and ResourceModifiedMessage
* ResourceDeliveryMessage and ResourceModifiedMessage tests
* fixed issue with test missing partitionHelperSvc mocked bean
* Added tests and implemented SubscriptionMatchingSubscriber for partition aware subscription
* modified implementation of partitionId in CanonicalSubscription
* Moved PartitionablePartitionId, and refactored all calls to getUserData(Constants.RESOURCE_PARTITION_ID) and setUserData(Constants.RESOURCE_PARTITION_ID)
* Revert "Moved PartitionablePartitionId, and refactored all calls to getUserData(Constants.RESOURCE_PARTITION_ID) and setUserData(Constants.RESOURCE_PARTITION_ID)"
This reverts commit fe40fb9733.
* Got added partitionId to subscriptions, added changes to make SubscriptionMatchingSubscriberTest work
* added SubscriptionTriggering test, also added partition support to subscriptionLoader
* Changed implementation for storing partition id of subscriptions from messages, refactored tests to new implementation
* added all subscription systemRequestDetails with all partition to subscription reader
* refactored a generic system request details with default all partition request
* Added test for dao subscriptions, fixes to get the test working
* added partition support for latest version delivery
* added doc changes and changelog for multitenancy subscription
* cleanup and added partitioned subscription manually trigger test
* fixed mocked subscriptionDao
* added package-info for subscription module
* some code review changes
* removed AllPartitionSystemRequestDetail, added new text for multitenant subscription
* renamed method for code review
* version bump to 5.7.0PRE7
Co-authored-by: Michael Buckley <michael.buckley@smilecdr.com>
Co-authored-by: Ken Stevens <khstevens@gmail.com>
Co-authored-by: Long Ma <long@smilecdr.com>
Co-authored-by: Steven Li <steven@smilecdr.com>
* move mail config to mail service constructor
* update subscription email sender
* added license text
* [2669] bump HAPI version to 5.7.0-PRE6-SNAPSHOT
* add changelog
* First cut at multithreading bundle validation
* javadoc
* change test to a red-green test
* add daoconfig
* move bundle concurrency config from validationsupport to request validator
* Moved concurrent details from ValidationOptions to FhirValidator
* Remove bundleValidationThreadCount (it's set by creating an appropriate executor)
* validate threadpool prefixes
* FIXME cleanup
* reassign FIXMEs to JB
* only create the threadpool if it will be used
* Add the MdcTaskDectorator to all threadpools
* Add bundle path to concurrently validated bundle resource validation messages.
* Add test to check bundle entry path being added to validation messages
* Threaded validation of contained resources initial checkin
* FHIR validator refactored to maintain correct bundle path and async validation task
* Refactored to use validation complete invoke callback to support raw string resource or resource object
* switch from XML to JSON
* # Conflicts:
# hapi-fhir-base/src/main/java/ca/uhn/fhir/validation/FhirValidator.java
# hapi-fhir-validation/src/test/java/org/hl7/fhir/r4/validation/FhirInstanceValidatorR4Test.java
* Updated to use new HAPI-FHIR version
* merge 'validate_contains_change' into issue-3144-multi-threaded-bundle-validation
* Updated to use new HAPI-FHIR version
* Revert "Updated to use new HAPI-FHIR version"
This reverts commit 910b7f676b.
* Revert "merge 'validate_contains_change' into issue-3144-multi-threaded-bundle-validation"
This reverts commit c7d60fc3e6.
* Revert "Updated to use new HAPI-FHIR version"
This reverts commit aa8a7af455.
# Conflicts:
# hapi-fhir-validation/src/test/java/org/hl7/fhir/r4/validation/FhirInstanceValidatorR4Test.java
* Fix Test dependencies after reverting changes.
* merge 'validate_contains_change' to 2564-contained-resource-validation-flag
* Disable threadpool prefix name validate check.
* merge master
* fix thread regexp
* pre-review cleanup
* change log
* fix test
* revert to XML as default validation serialization
* fix test
* fix test
Co-authored-by: Jaison B <jaisonb@gmail.com>
* Synthea perf work
* Cleanup
* Work on performance
* Ongoing perf work
* More perf work
* Perf work
* Test fixes
* Updates
* Test fix
* Test fixes
* Clean up tests
* Test cleanup
* Cleanup
* Add test
* One more perf tweak
* Fixes
* Add changelog
* Add changelog
* License header update
* License header updates
* issue-1134 initial work on adding retry handler
* issue-1134 subdscription creation fixes
* issue-1134 update the test
* issue-1134 retry policy off canonicalsubscription
* issue 1134 base subscription changes
* add failing test
* issue 1134 retry configs on base parameters
* issue 1134 passing forward params
* issue-1134 added more tests for making sure subscriptions create their channels correctly
* issue-1134 updates to get the retry to channels
* issue-1134 updating channel factory
* issue-1134 remove the dlq since it's not defineable
* issue-3120 changelog added
* issue-3120 cleaning up
* issue-3120 test fix
* issue-3120 review fixes
* issue-3120 fixed bad master merge
* issue-3120 updates for new release
Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
Co-authored-by: Ken Stevens <khstevens@gmail.com>
* issue-1134 initial work on adding retry handler
* issue-1134 subdscription creation fixes
* issue-1134 update the test
* issue-1134 retry policy off canonicalsubscription
* issue 1134 base subscription changes
* add failing test
* issue 1134 retry configs on base parameters
* issue 1134 passing forward params
* issue-1134 added more tests for making sure subscriptions create their channels correctly
* issue-1134 updates to get the retry to channels
* issue-1134 updating channel factory
* issue-1134 remove the dlq since it's not defineable
* issue-3120 changelog added
* issue-3120 cleaning up
* issue-3120 test fix
* issue-3120 review fixes
* issue-3120 fixed bad master merge
Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
Co-authored-by: Ken Stevens <khstevens@gmail.com>
* Add failing test
* Tidy test for new tokenparam style
* Add removeByQualifier to SearchParameterMap
* Add new property binder for search terms
* Member move, add test for new parser
* whoops missed an add
* Passing test.
Introduced SearchParamTextWrapper to give a type for the binder.
Enhance FulltextSearchSvcImpl to understand TokenParams as well as StringParams.
* Add more to test, change analyzer for template
* Optimize imports
* Minor refactoring
* added eq and gt test cases
* added missing gt search param tests
* Search all CodeableConcepts
* Extract new LuceneRuntimeSearchParam to mediate indexing and searching.
* Fix up broken test with mock
* Add support for identifier type
* Bump for special maven package
* fix-me to todo
* Prevent version check
* FIx version for test
* DSTU2 does not support fluent path
* Don't accidentally strip SPs in DSTU2
* added lt tests
* Add some high-bound test cases
* More test cases and new compressed format
* More test cases
* More test cases
* Re-order columns for easier reading
* Comments
* Cleanup and comments
* Make support :text for all token SPs
* Make support :text for all token SPs
* Fixed incomplete date issue
* Set to the last millisecond 23:59:59.999
* Disabled 4 failed JVM/TZ related test cases
* add test for failing batch bug, remove bounding on queue
* remove parallelism from non-get batch operations
* Fix count for test
* New index layout
* Notes and cleanup
* Notes and cleanup
* remove the jetbrains
* Demote FIXME to WIP
* Change :text search to be lucene simple search and use standardAnalyzer.
* Change :text to require * for prefix match.
* add prefix negative test
* Bump version
* Added changelog
* Add more search parameters to lucene indexing (#3000)
* dirty the branch
* Add oversight to changelog
* Start using jpa indexing for es
* Correct negative search test
* Update templates, index tokens, write new test. TODO gotta modify SB
* Add test for code token search
* add comment
* Flesh out token search
* minor cleanup and comments
* Extract index handling from FulltextSearchSvcImpl
* D'oh. Actually use elastic to search.
* Move supported query check closer to query builder
* String search test before activating lucene
* Add string:exact to hibernate search.
* Add string:contains search
* cleanup
* demote fixmes to allow build to run.
* Add unmodified string search.
* empty stubs for quantity and reference
* Ignore magic search params in lucene index
* Support full-text on all FHIR releases
* Include reference lookups in ES
* Fix and/or logic in _text
* Test for string with space
* Cherry-pick the NoFT fix
* Disable advanced indexing in a bunch of tests of emitted sql.
* Stub changelog
* Fix :missing flag
* Move DaoConfig up to BaseJpaTest to share teardown
* Disable new indexing for partition test
* Add a different analyzer and field for default string search vs text
* checkstyle for a pre-build
* Index full prefix for string search when using Lucene
* Update hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/5_6_0/3000-Extend-lucene-indexing.yaml
Co-authored-by: Tadgh <tadgh@cs.toronto.edu>
* Update hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/dao/HibernateSearchQueryBuilder.java
Co-authored-by: Tadgh <tadgh@cs.toronto.edu>
* Review feedback.
Co-authored-by: Tadgh <garygrantgraham@gmail.com>
Co-authored-by: Tadgh <tadgh@cs.toronto.edu>
* Remove double-check
* Version bump
* Fix typo
Co-authored-by: Michael Buckley <michael.buckley@smilecdr.com>
Co-authored-by: Long Ma <longma@Longs-MacBook-Pro.local>
Co-authored-by: Frank Tao <frankjtao@gmail.com>
Co-authored-by: michaelabuckley <michaelabuckley@gmail.com>
* Create index entries for outbound references of contained resources
* build query for chained reference
* fix case where the contained reference is an explicit id rather than a continued chain
* fix contained index to use path names not search param names
* make qualified search work
* cleanup and changelog
* recurse while creating indexes on contained resources
* double link both contained
* longer contained subchains
* adding some failing test cases to illustrate the limitations of qualified searches
* clean up merge cruft
* changelog
* create recursive resource links
* add test coverage for a more complicated case
* changelog
* remove unnecessary check for _contained flag
* fix broken tests
* Create index entries for outbound references of contained resources
* build query for chained reference
* fix case where the contained reference is an explicit id rather than a continued chain
* fix contained index to use path names not search param names
* make qualified search work
* cleanup and changelog
* recurse while creating indexes on contained resources
* double link both contained
* longer contained subchains
* adding some failing test cases to illustrate the limitations of qualified searches
* clean up merge cruft
* changelog
* Create index entries for outbound references of contained resources
* build query for chained reference
* fix case where the contained reference is an explicit id rather than a continued chain
* fix contained index to use path names not search param names
* make qualified search work
* cleanup and changelog
* code review
* fix broken tests
* Add tests. Add implementation in R4. Add changelog
* Wip partial fixup of other STU versions
* Updated usages of new patient type everything api
* Partial commit of incomplete documentation
* Remove docs
* Cut to token instead of string
* Bump hapi version due to breaking api changes
* Migration finished
* Work on LOBS
* Add to ClobMigrated annotation
* Deal with lobs
* Work on new approach
* HAPI FHIR version bump
* Add changelog
* Add license header
* Fix intermittent
* CLeanup
* Cleanup
* Add failling test, works on #2843
* Partial hack to get this working
* Move implementation into pre-processing
* Add changelog
* Move implementation
* Minify Test
* Remove fixmes
* Tidying
* Ensure this only operates on STU versions which support canonical
Mark Iantorno
Tadgh
Ken Stevens
Jaison Baskaran
JasonRoberts-smile
longma1
James Agnew
michaelabuckley
StevenXLi
Tadgh
Johnson Lu
TipzCM
VK-SMILECDR