mirror of
https://github.com/discourse/discourse-ai.git
synced 2025-02-07 12:08:13 +00:00
830cc26075
* FEATURE: Add metadata support for RAG You may include non indexed metadata in the RAG document by using [[metadata ....]] This information is attached to all the text below and provided to the retriever. This allows for RAG to operate within a rich amount of contexts without getting lost Also: - re-implemented chunking algorithm so it streams - moved indexing to background low priority queue * Baran gem no longer required. * tokenizers is on 4.4 ... upgrade it ...
9 lines
241 B
Ruby
9 lines
241 B
Ruby
# frozen_string_literal: true
|
|
|
|
class AddMetadataToRagDocumentFrament < ActiveRecord::Migration[7.0]
|
|
def change
|
|
# limit is purely for safety
|
|
add_column :rag_document_fragments, :metadata, :text, null: true, limit: 100_000
|
|
end
|
|
end
|