Skip to main content
Introducing packages.sweber.dev
Documentation menuJavaScript API

JavaScript API

loadIndex, Cosine, buildIndex and the helpers.

Browser

loadIndex(url, options?)

Loads cosine-index.json and the vector file next to it and returns a Cosine instance.

OptionDefault
loadModellazylazy loads the model on warmup() or the first search (not when the browser asks to save data), eager right away, never not at all
embedderfrom the indexYour own Embedder, or false for keyword search only
minSimilarityfrom the indexDrop semantic hits below this cosine similarity
boostfrom the indexRanking weights per URL path, e.g. { "/docs/api": 1.5 }, replacing the ones stored in the index
synonymsfrom the indexSynonym groups, e.g. [["login", "sign-in"]], replacing the ones stored in the index
fetchglobalThis.fetchCustom fetch

Cosine

Member
search(query, options?)Promise<SearchResult[]>. mode: hybrid (default; keyword results until the model is ready, does not wait), lexical, semantic (waits for the model). limit (8), groupByPage (false), scope (URL paths to search in, a string or an array)
facets(query, options?)Promise<{ path, count }[]>: the results of a query counted per section of the site, most first, e.g. /docs/guides 12, /docs/api 5. depth (path segments per facet, default 1), limit (results counted, 200), plus mode and scope. Use a path as scope to search only there
searchLexical(query, options?)Synchronous keyword search
warmup()Loads the model. Never rejects; on failure status becomes model-failed
statuslexical, loading-model, ready, model-failed
onStatus(listener)Subscribe to status changes, returns an unsubscribe function
chunksAll chunks of the index

CosineGroup

loadIndexes(urls, options?) loads several indexes and returns a CosineGroup; new CosineGroup(members, { weights }) groups indexes you loaded yourself. It has search, searchLexical, facets, warmup, status, onStatus and chunks like Cosine, merges results by rank, and adds index (position of the index) to each result. weights ranks an index higher. See Several indexes.

SearchResult

interface SearchResult {
  chunk: { id: number; doc: string; url: string; title: string; headings: string[]; text: string };
  score: number;
  matchedBy: Array<"lexical" | "semantic">;
  snippet: string;
  highlights: Array<[start: number, end: number]>; // ranges in snippet
}

highlightParts(snippet, highlights) splits a snippet into { text, match } parts for rendering.

Build (Node and browser)

Function
buildIndex(documents, options)Chunks and embeds { id, url, title, content } documents. Returns { manifest, vectors, reused, embedded }. synonyms stores synonym groups in the index, boost stores ranking weights per path, previous ({ manifest, vectors } of the last build) reuses the vectors of unchanged sections
transformersEmbedder(options)Embedder on top of transformers.js. model (english, multilingual or an id), dtype (q8), device, queryPrefix, passagePrefix, load, onProgress
chunkMarkdown(document, options)Splits one Markdown page into chunks
htmlToMarkdown(html)Extracts the main content of an HTML page

From @sweberdev/cosine/node: readDocs(dir, options), writeIndex(index, outDir, { shardSize }), buildDirectory(dir, outDir, options), readIndex(dir) (the last build, for previous), loadIndexFile(path, options).

Custom embedder

Anything with this shape works, e.g. a call to your own embedding API at build time:

interface Embedder {
  model: string;
  embed(texts: string[], kind: "query" | "passage"): Promise<Float32Array[]>;
}

Reading and writing the index yourself

parseManifest(json) turns the content of a cosine-index.json (format 1 or 2) into a manifest with all sections. serializeManifest(manifest) writes the compact format 2. indexFiles(builtIndex, { shardSize }) returns the files of an index as { name, data } (the manifest, the vector files and, with shardSize, the shards), for build tools that emit files themselves. See Migrating to 2.0.