Interface for caching results from embedding models.

The interface allows works with any store that implements the abstract store interface accepting keys of type str and values of list of floats.

If need be, the interface can be extended to accept other implementations of the value serializer and deserializer, as well as the key encoder.

const underlyingEmbeddings = new OpenAIEmbeddings();

const cacheBackedEmbeddings = CacheBackedEmbeddings.fromBytesStore(
underlyingEmbeddings,
new ConvexKVStore({ ctx }),
{
namespace: underlyingEmbeddings.modelName,
},
);

const loader = new TextLoader("./state_of_the_union.txt");
const rawDocuments = await loader.load();
const splitter = new RecursiveCharacterTextSplitter({
chunkSize: 1000,
chunkOverlap: 0,
});
const documents = await splitter.splitDocuments(rawDocuments);

let time = Date.now();
const vectorstore = await ConvexVectorStore.fromDocuments(
documents,
cacheBackedEmbeddings,
{ ctx },
);
console.log(`Initial creation time: ${Date.now() - time}ms`);

time = Date.now();
const vectorstore2 = await ConvexVectorStore.fromDocuments(
documents,
cacheBackedEmbeddings,
{ ctx },
);
console.log(`Cached creation time: ${Date.now() - time}ms`);

Hierarchy

  • Embeddings
    • CacheBackedEmbeddings

Constructors

Properties

caller: AsyncCaller

The async caller should be used by subclasses to make any async calls, which will thus benefit from the concurrency and retry logic.

documentEmbeddingStore: BaseStore<string, number[]>
underlyingEmbeddings: EmbeddingsInterface

Methods

  • Embed a list of texts.

    The method first checks the cache for the embeddings. If the embeddings are not found, the method uses the underlying embedder to embed the documents and stores the results in the cache.

    Parameters

    • documents: string[]

    Returns Promise<number[][]>

    A list of embeddings for the given texts.

  • Embed query text.

    This method does not support caching at the moment.

    Support for caching queries is easy to implement, but might make sense to hold off to see the most common patterns.

    If the cache has an eviction policy, we may need to be a bit more careful about sharing the cache between documents and queries. Generally, one is OK evicting query caches, but document caches should be kept.

    Parameters

    • document: string

      The text to embed.

    Returns Promise<number[]>

    The embedding for the given text.

  • Create a new CacheBackedEmbeddings instance from another embeddings instance and a storage instance.

    Parameters

    • underlyingEmbeddings: EmbeddingsInterface

      Embeddings used to populate the cache for new documents.

    • documentEmbeddingStore: BaseStore<string, Uint8Array>

      Stores raw document embedding values. Keys are hashes of the document content.

    • Optionaloptions: {
          namespace?: string;
      }
      • Optionalnamespace?: string

        Optional namespace for store keys.

    Returns CacheBackedEmbeddings

    A new CacheBackedEmbeddings instance.