AI for Engineering Knowledge Management
Engineering documents put the real answer in the figure, not the paragraph. What an AI assistant has to do to read it, what it still gets wrong, and how to prepare your documents.
·
⏱
8 min read
Dr. Maor Farid
Maor Farid is the Co-Founder and CEO of Leo AI, the first AI platform purpose-built for mechanical engineers. He holds a PhD in Mechanical Engineering and completed postdoctoral research at MIT as a Fulbright fellow. A Forbes 30 Under 30 honoree and former AI researcher and Mechanical Engineer in an elite military intelligence, Maor leads Leo AI’s mission to transform how engineering teams design better products faster.
BOTTOM LINE
In engineering documents, the answer is frequently in the figure and the text is only a pointer to it. Any assistant built on text extraction alone will hand back that pointer, quietly skip scanned pages, or fill the gap with a general-purpose guess, and all three look like a working answer. Reading the figure means resolving the reference, rendering the region properly, reading geometry and callouts together, keeping the revision attached, and citing a location someone can open. Even then, low-resolution scans, redlines, figures that disagree with the text, and missing context still need a person. The practical move is to keep documents in a form where figures remain readable, and to expect an assistant to show its source rather than paraphrase a drawing into prose.
A work instruction tells you to install the seal per Figure 6. The torque values sit in a table that was pasted in as an image. The pin assignment lives in a connector illustration someone scanned a decade ago. Engineers read documents like this every day, and the part that carries the answer is very often the part that is not text at all.
That gap matters more now that teams point an AI assistant at their document set and expect an answer rather than a list of search hits. A system that reads only the extracted text of a PDF will return the sentence that points at the figure, which is the one thing the engineer already had. The figure itself never enters the answer.
This post covers why engineering information keeps ending up in figures, what text-only retrieval returns instead, what an assistant actually has to do to answer from a drawing or diagram, where that still falls short, and how to set your documents up so the answer is reachable in the first place.
Why the Answer Keeps Ending Up in the Figure
This is not sloppy authoring. Engineering documents carry information that prose handles badly, and the figure is the correct place to put it. Four categories come up again and again:
Orientation and handedness. Which way a seal lip faces, which side of a joint gets the fillet weld, which direction flow runs through a manifold. A sentence can say it, but the section view is what people trust.
Spatial relationship. Datum locations, clearance envelopes, the plane a section was cut on, the specific hole in a pattern that a note applies to. Position is the content.
Tabulated data captured as a picture. Torque charts, material property tables, and revision blocks routinely arrive inside a document as an image rather than as live text, especially in supplier documentation and anything that passed through a scanner.
Sequence. Assembly steps, disassembly order, and inspection routing are drawn because the ordering is easier to see than to read.
Once the information is in the figure, the surrounding text turns into a pointer. Refer to Detail B. See Figure 4. Torque per Table 3. The prose is navigation, and the payload sits somewhere the prose only names.
Standards work the same way. Geometric tolerancing conveys meaning through symbol geometry and the order of segments inside a frame, not through sentences, which is why reading a feature control frame is a skill in its own right. A tolerance zone is a shape. Flatten it into text and the shape is what you lose.
IN PRACTICE
It surfaces the relevant internal material, previous design decisions, past calculations, and backs everything with a cited source I can actually click on and verify.
- Yuval F., Clalit
What Text-Only Retrieval Gives You Instead
Most document assistants are built on the same pipeline. Pull the text layer out of the PDF, split it into chunks, embed the chunks, retrieve the closest ones, and hand them to a language model. Everything that is not in the text layer never enters that pipeline. Figures are dropped, or at best reduced to a caption string.
That produces three distinct failure modes, and they look different from the outside:
The pointer answer. You ask which way the seal faces and the assistant returns install the seal per Figure 6, sometimes with a tidy summary of the paragraph around it. The answer is technically grounded in the document and completely useless.
The silent gap. A scanned page has no text layer at all, so the document reads as nearly empty and ranks low against every query. The assistant answers instead from some other document that did have text, which may be older, superseded, or from a different program. Nothing in the response signals that the authoritative page was skipped.
The confident guess. With the figure invisible, the model falls back on general knowledge and supplies a plausible default. Seal lips usually face the pressure side. Usually is doing a lot of work in that sentence, and the failure is quiet because the answer sounds specific.
Teams often describe this as the assistant not working on their documents, without being able to say why. The pattern underneath is usually that the highest-value pages were also the most visual, so the tool was effectively blind to exactly the content worth asking about. It is the same reason general-purpose engineering knowledge base software tends to disappoint mechanical teams: the format assumptions were written for text documents.
What It Takes to Answer From a Figure
Answering from a figure is a chain, and every link has to hold. The reference has to resolve, so that see Figure 6 maps to a specific region of a specific page rather than to a caption sitting somewhere in the text. The region has to be rendered at a usable resolution rather than inherited from a thumbnail. The geometry and the callouts have to be read together, because a leader line pointing at the wrong feature changes the meaning entirely. The figure has to stay attached to its own document and revision, so an answer pulled from a superseded drawing is recognisable as such. And the response has to come back with a citation specific enough to open, not a document-level reference that leaves someone scrolling.
This is the layer Leo is built for. Leo is an AI assistant for mechanical engineers, trained on more than one million pages of standards, books, and articles, and it connects to the knowledge an organisation already holds rather than asking anyone to move it. Leo offers integrations with leading PDM and PLM platforms (SolidWorks PDM, Autodesk Vault, PTC Windchill, Siemens Teamcenter, Arena PLM, and others), along with local and network directories and ERP. It sits on top of those systems as an intelligence layer rather than replacing them, which matters here because the documents worth reading are already in the vault and already under revision control.
The value driver is narrow and real. An engineer who cannot get an answer out of a figure does one of three things: asks a colleague who has read that document before, re-derives the answer from first principles, or guesses. The first two cost hours and depend on who happens to be available, which is the same dynamic that makes tribal knowledge so expensive when people move on. The third costs a build. Returning the figure with a citation collapses all three into a question that takes a minute.
What the Figure Still Cannot Tell You
Reading the figure removes one failure mode. It does not remove judgement, and the honest limits are worth stating plainly.
Resolution and provenance. A bitmap scanned at low resolution, or a photocopy of a photocopy, loses thin leader lines and decimal points before it loses anything else. A dimension that reads 12.5 and a dimension that reads 125 can be one bad pixel apart. The right behaviour is to flag low confidence and show the crop, not to commit to a digit.
Redlines and markups. Hand annotations on a released print often supersede what is printed underneath, and nothing on the page says so in machine-readable form. Whether the marked value governs is a question about your change process, not about the image.
Figures that contradict the text. This is common, and it usually means a revision updated one and missed the other. Surfacing the conflict is genuinely useful. Deciding which one governs is an engineering call with a named person attached to it.
Context that is not on the page. A fastener pattern drawn without its load case, a fit called out without the mating part, a chart without its units. The figure is complete only relative to a document someone has to supply.
The useful behaviour across all four is the same. Show the figure, cite where it came from, and say what is uncertain, rather than paraphrasing the picture into a confident sentence that nobody can check. An answer an engineer can verify in ten seconds is worth more than an answer that is right slightly more often and opaque every time.
Making Your Documents Answerable
Most of what determines whether a figure can be read back was decided when the document was produced. Five habits do most of the work:
Keep the vector original. Export the PDF from the authoring tool instead of printing and scanning it. A vector figure keeps its line geometry and its text as text, and it survives being zoomed.
Run recognition over legacy scans, and keep the image beside the recovered text rather than replacing it. The text layer makes the page findable; the image is still what the answer has to come from.
Number and caption figures so the reference resolves. See Figure 6: seal orientation, lip toward pressure side is a caption that does work. See below is not.
Put the revision identifier on the page itself, not only in the file name. File names get copied, renamed, and stripped; the page travels with its content.
Retire superseded copies or mark them unmistakably. A retrieval system cannot infer which of four near-identical PDFs is current, and neither can a new engineer.
These are the same habits that make engineering document control work at all, which is convenient: the investment pays twice. Teams moving toward model-based definition change the shape of the problem rather than removing it, since the annotations then live in the model and the question becomes whether a retrieval layer can read product manufacturing information out of it. The figure does not go away. It just stops being flat.
FAQ
Ask your documents, figures included
See how Leo reads the engineering documents your team already has
Leo connects to your PDM, PLM, and file directories, reads what is already there, and answers with a citation you can open and check against the source page.
Schedule a Demo →
#1 New AI Software Globally - G2 2026
Enterprise-grade security
Trusted by world-class engineering teams
