CosineLive demo, free and Pro.
Everything below runs in your browser: the index of the Cosine and Surjection docs, keyword search, and, once you load it, the embedding model that searches by meaning. There is no search server.
Your queries are not sent anywhere. Loading the model downloads about 23 MB from Hugging Face once; your browser caches it.
Search
The search field (free, MIT)Try it, then load the model.
This is <cosine-search>, the web component from @sweberdev/cosine, on an index of 44 documentation pages. Keyword results appear on the first keystroke, and typos are forgiven: try “instalation” or “acessibility”. The buttons under the field filter by section, synonyms are stored in the index (“a11y” finds the accessibility pages), and search syntax works: try "exact phrase" or -excluded words. Load the model and the same field also finds sections that use other words than your query. Press / to focus it.
Keyword search only. Load the model to search by meaning.
In a real site the section vectors are computed once at build time with `cosine build`. This demo computes them in your browser after the download, which takes a few seconds.
Keyword vs. meaning
Pick a question or type your own. The left column is classic keyword search (BM25), as in most static-site search tools. The right column is Cosine's hybrid ranking: keywords and embeddings fused, so a question finds the section that answers it.
Example questions
Insights
Search insights (Pro)What your docs are missing.
@weber-development/cosine-insights records which searches find nothing and which results nobody opens, without cookies, ids or IP addresses, and turns them into a report. The example below was made from a month of made-up searches on a docs site.
npx cosine-insights report --log data/searches.ndjson --out insights.htmlFree 2.0
Bigger sites, more languages, fewer files to worry aboutCosine, free.
Cosine 2.0 makes the MIT package ready for very large docs: a smaller index format, shards, and 4-bit vectors. It also finds words in Chinese, Japanese and Thai, and ships integrations for Astro (Starlight), Docusaurus and VitePress.
Shards for large sites
Split the index into several files that the browser loads in parallel. Results rank exactly like one file, and unchanged shards keep their name so your CDN can cache them.
Build with shards
cosine build docs --out public/cosine --lexical-only --shard-size 10Real output (13 pages, lexical build) and the manifest that lists the shards
Indexed 13 pages as 25 chunks in 0.0 s (lexical only)
public/cosine/cosine-index.json
public/cosine/cosine-shard-0.json
public/cosine/cosine-shard-1.json
public/cosine/cosine-shard-2.json
{"version":2,"model":null,"dimensions":0,"createdAt":"...","vectors":null,"shards":[{"file":"cosine-shard-0.json","chunks":10},{"file":"cosine-shard-1.json","chunks":10},{"file":"cosine-shard-2.json","chunks":5}]}Search in Chinese, Japanese and Thai
Cosine 2 splits scripts without spaces into words with Intl.Segmenter, so a search for a short word finds longer phrases and highlights it.
Search the shard index
cosine search public/cosine 東京 --mode lexical --limit 1Real output
1. Tokyo (lexical)
/tokyo
東京 東京タワーは東京の有名な観光地です。4-bit vectors
Store each vector value in 4 bits. The vector file is half the size, at a small cost in ranking quality.
Build with smaller vectors
cosine build docs --shard-size 2000 --vector-bits 4Docusaurus, Astro and VitePress integrations
Add the plugin and the index is built with your site. The same package exports /astro and /vitepress.
docusaurus.config.js
plugins: [["@sweberdev/cosine/docusaurus", { model: "english" }]],Real output comes from @sweberdev/cosine 2.0.0 from npm, using a keyword-only build. The 4-bit vectors and the integrations are shown from the docs.
Read the docsPro 2.1.0
Search for every docs framework and languageCosine Pro.
Pro 2.1.0 adds ready-made search for Docusaurus, VitePress and Astro Starlight, and a separate index and model for each language of your site. Visitors only download the model of the language they are reading.
One index per language
Give each language its own model, for example the small English model for English pages and the multilingual one for German. The language of a page comes from front matter, the file name or the folder.
build.ts
import { readDocs } from "@sweberdev/cosine/node";
import { buildLanguageIndexes, writeLanguageIndexes } from "@weber-development/cosine-i18n";
const docs = await readDocs("docs", { baseUrl: "/docs" });
const build = await buildLanguageIndexes(docs, {
languages: { en: "english", de: "multilingual", fr: "multilingual" },
});
await writeLanguageIndexes(build, "public/cosine", { shardSize: 500 });Docusaurus pluginBeta
Replaces the navbar search with Cosine and builds the index of every locale after the build, with an embedding cache so rebuilds only embed changed sections.
docusaurus.config.js
themes: [
[
"@weber-development/cosine-docusaurus",
{ models: { en: "english", de: "multilingual" } },
],
],
themeConfig: {
navbar: {
items: [{ type: "search", position: "right" }],
},
},VitePress search fieldBeta
Wrap your config and use the theme: the search field replaces the built-in search, the index is built in dev and in build, and VitePress locales map to languages.
.vitepress/config.ts
import { defineConfig } from "vitepress";
import { withCosine } from "@weber-development/cosine-vitepress";
export default withCosine(defineConfig({ title: "My docs", base: "/docs/" }), {
model: "english",
});Astro and StarlightBeta
A Starlight plugin swaps the Pagefind search for Cosine, follows the light and dark theme and picks the index of the page language.
astro.config.mjs
import starlight from "@astrojs/starlight";
import cosineStarlight from "@weber-development/cosine-astro/starlight";
import { defineConfig } from "astro/config";
export default defineConfig({
integrations: [
starlight({
title: "Docs",
plugins: [cosineStarlight()],
}),
],
});This demo cannot show the framework plugins: they need your own Docusaurus, VitePress or Astro site. The search field of the plugins was not yet tried in a real browser, hence Beta.
Read the docsNext
Add it to your docsTwo commands and one tag.
Build the index with npx cosine build docs --out public/cosine, add <cosine-search index="/cosine/cosine-index.json"> to your layout, done. Cosine Pro builds the index in every Vite build and adds search insights.