How Memo builds a knowledge graph from your documents

When you upload a document to Memo, what comes back a little later is not a stored file — it is a set of new nodes and edges woven into your knowledge graph. Between those two moments sits a pipeline of six steps, each doing one job well. This post walks through all six in plain language, and, more importantly, explains why each one is there. The steps are easy to list; the reasoning behind them is what makes the graph worth having.

The stages are: upload, chunk, embed, extract, resolve, store. Let's take them in order.

Upload — your bytes take the short route

The first decision Memo makes is to get out of the way. When you add a document, the bytes travel directly from your browser to secure object storage through a one-time upload URL. Memo's services coordinate the work but never sit in the byte path.

This is a deliberate architectural choice, not an accident. Your files take the shortest route to storage, which is faster and means document contents are not flowing through service compute that does not need to see them. Memo's services act as a control plane — they orchestrate, they do not ferry. Everything that follows happens once the bytes have safely landed.

Chunk — turning a document into passages

A whole document is the wrong unit to reason over. It is too big to search precisely and too big to feed cleanly to a model. So the first thing Memo does after upload is split the text into passages.

How that text is obtained depends on the source. Markdown, HTML and plain text are chunked natively. PDFs are handled with PyMuPDF, with a poppler pdftotext fallback for the awkward cases, and everything is UTF-8 sanitised so stray encodings don't poison later steps.

Why passages rather than whole files? Because relevance lives at the passage level. When you later search or ask a question, you want the specific paragraph that answers you — and the citation that points at it — not a whole report you then have to re-read. Chunking is what makes precise retrieval and precise citations possible.

Embed — search by meaning, not by keyword

Each passage is then embedded with OpenAI's text-embedding-3-small model, which turns it into a 1536-dimension vector — a numeric fingerprint of its meaning. Your search queries are embedded with the same model, so a query and the passages it matches are described in the same terms.

This is the difference between searching by meaning and searching by keyword. Keyword search only finds the words you typed; ask about "staff turnover" and a passage that talks about "employees leaving the company" stays invisible, because it shares no words with your query. Semantic embeddings place related ideas near each other regardless of the exact wording, so Memo can surface the right passage even when it phrases things differently from you.

Extract — finding the entities and relationships

Embeddings make passages findable, but a graph needs structure: things, and the connections between them. So Memo passes the text to a language model that extracts entities — people, organisations, places and concepts — and the typed relations that link them.

This is where a pile of documents starts becoming a graph. Instead of just "a passage that mentions a name", you get a named entity and an edge describing how it relates to another entity. The model reads for the kind of structure a person would notice on a careful read, and records it in a form the graph can hold.

Resolve — one entity, not many copies

Extraction alone has a problem: the same real-world thing gets mentioned in many documents, and each mention looks, at first, like a separate discovery. Left unresolved, your graph would fill with near-duplicates — the same organisation appearing as a dozen disconnected nodes, its story scattered and its connections invisible.

Entity resolution fixes this. Memo resolves extracted entities to canonical nodes, so the same entity across different documents collapses into one. Mention an organisation in five reports and you get a single node with five sources behind it.

This is the step that makes the graph genuinely useful rather than merely large. The whole promise of a knowledge graph is that it connects what you know — and connection only happens when "this organisation here" and "that organisation there" are understood to be the same organisation. Without canonical resolution you have a heap; with it, you have a graph.

Store — Postgres and pgvector, collapsing to canonical nodes

Finally everything is stored: the passages and their embeddings, the entities, and the relations. Memo uses Postgres with the pgvector extension, which lets the same database hold both the structured graph and the vectors that power semantic search.

When you read from the graph — browsing it, searching, or asking a question — Memo collapses results down to those canonical nodes. So a search that touches five documents about the same organisation reports back through one coherent node, with the underlying sources still attached. Storage here is not just a resting place; it is what keeps reads consistent with the resolution work done earlier.

Why the whole thing hangs together

Each step earns its place. Direct upload keeps your bytes out of the byte path. Chunking makes retrieval and citations precise. Embeddings let you search by meaning. Extraction turns prose into structure. Resolution turns structure into a connected graph. Storage keeps reads honest. Pull any one out and the result is worse — a graph that is slower, vaguer, more duplicated, or harder to cite.

To go deeper on the graph itself, see the knowledge graph. For how queries and cited answers work once the graph is built, see search and ask.