What "your own data" actually means

"Point it at our data" is where most projects begin and where a surprising number quietly stall. The phrase implies a single place. In practice a company's knowledge is spread across systems with different shapes, different owners and — the part that catches people — different rules about who may read what.

The four shapes

Almost everything falls into one of four, and each is a different engineering problem.

Documents. Policies, contracts, specs, slide decks. Unstructured text, often in PDFs that were scanned rather than exported. The hard parts are parsing and chunking.

Databases. Orders, inventory, customers. Structured, authoritative, and the worst possible candidate for embedding. You do not want a vector search over your orders table — you want the model to write a query. Treating structured data as text is one of the most common early mistakes, and it produces an assistant that is confidently approximate about numbers.

APIs. The ticketing system, the CRM, the calendar. Live, authoritative, and best called at question time rather than copied.

Conversations. Email, chat, support threads. Enormous, full of genuine institutional knowledge, and a minefield of personal data and half-decisions that were later reversed.

The practical consequence: an assistant worth using needs at least two of these paths, and the common tutorial — embed a folder of PDFs — covers one.

The inventory

Before writing code, produce a table. For each source: what it holds, who owns it, who may read it, how it is accessed, how often it changes, and whether a question has ever actually needed it.

That last column is the useful one. It is normal for a team to spend three weeks ingesting a system nobody ever asks about, because it was easy to connect.

A quick way to produce the inventory honestly: take the twenty most recent real questions people asked a colleague over chat, and for each one write down where the answer actually lived. The sources that appear are your scope. The ones that do not appear are version two.

Permissions come first, and here is why

The temptation is to index everything now and add permissions later. It does not work, for a specific structural reason.

Permission information has to ride along with each chunk, stamped at ingestion. Retrofitting means re-ingesting everything, and in the meantime the system either leaks or returns nothing.

This connects to the permission leak from the previous chapter. The leak is not caused by a careless prompt. It is caused by a chunk that does not know who owns it.

Map the permission model before the data model. If you cannot answer "who may read this document" for every source in the inventory, you are not ready to index.

The data you should refuse

Part of the inventory is deciding what never enters the system. Reasonable candidates: anything with health or payroll data, legal correspondence under privilege, and anything your contracts promise to keep in a particular jurisdiction.

Saying no to a source is a legitimate engineering decision and much easier before it is indexed than after. Write the refusals down with reasons, because you will be asked again in six months by someone who was not in the room.

Freshness, per source

Different sources tolerate different staleness. The returns policy changing twice a year needs a nightly sync at most. Inventory levels are wrong the moment they are copied, which is the argument for querying that database live rather than indexing it.

Decide this per source, in the inventory, rather than building one pipeline that treats a contract and a stock level the same way.

Key points

• Company knowledge comes in four shapes and each needs a different access path.
• Structured data should be queried, never embedded. Embedding numbers produces confident approximations.
• Build the inventory from real questions people actually asked, not from what is easy to connect.
• Permissions are stamped on chunks at ingestion; retrofitting means re-ingesting everything.
• Decide freshness per source. A contract and a stock level are not the same problem.