Gian Merlino 882ae9f002

Speed up composite key joins on IndexedTable. (#13516) ...

* Speed up composite key joins on IndexedTable.

Prior to this patch, IndexedTable indexes are sorted IntList. This works
great when we have a single-column join key: we simply retrieve the list
and we know what rows match. However, when we have a composite key, we
need to merge the sorted lists. This is inefficient when one is very dense
and others are very sparse.

This patch switches from sorted IntList to IntSortedSet, and changes
to the following intersection algorithm:

1) Initialize the intersection set to the smallest matching set from the
   various parts of the composite key.

2) For each element in that smallest set, check other sets for that element.
   If any do *not* include it, then remove the element from the intersection
   set.

This way, complexity scales with the size of the smallest set, not the
largest one.

* RangeIntSet stuff.

2023-02-17 22:01:01 -08:00

..

src

Speed up composite key joins on IndexedTable. (#13516)

2023-02-17 22:01:01 -08:00

pom.xml

merge druid-core, extendedset, and druid-hll into druid-processing to simplify everything (#13698)

2023-02-17 14:27:41 -08:00