Agustin Gonzalez 8e5048e643 Avoid memory mapping hydrants after they are persisted & after they are merged for native batch ingestion (#11123) ... * Avoid mapping hydrants in create segments phase for native ingestion * Drop queriable indices after a given sink is fully merged * Do not drop memory mappings for realtime ingestion * Style fixes * Renamed to match use case better * Rollback memoization code and use the real time flag instead * Null ptr fix in FireHydrant toString plus adjustments to memory pressure tracking calculations * Style * Log some count stats * Make sure sinks size is obtained at the right time * BatchAppenderator unit test * Fix comment typos * Renamed methods to make them more readable * Move persisted metadata from FireHydrant class to AppenderatorImpl. Removed superfluous differences and fix comment typo. Removed custom comparator * Missing dependency * Make persisted hydrant metadata map concurrent and better reflect the fact that keys are Java references. Maintain persisted metadata when dropping/closing segments. * Replaced concurrent variables with normal ones * Added batchMemoryMappedIndex "fallback" flag with default "false". Set this to "true" make code fallback to previous code path. * Style fix. * Added note to new setting in doc, using Iterables.size (and removing a dependency), and fixing a typo in a comment. * Forgot to commit this edited documentation message		2021-05-11 14:34:26 -07:00
..
src	Avoid memory mapping hydrants after they are persisted & after they are merged for native batch ingestion (#11123)	2021-05-11 14:34:26 -07:00
pom.xml	Fix idempotence of segment allocation and task report apis in native batch ingestion (#11189)	2021-05-07 14:29:48 -07:00