Optical Character Recognition (OCR) is software that looks at an image of text — a scan, a photo, a fax — and produces the actual characters as machine-readable data. To a computer, a scanned page is nothing but a grid of pixels. It has no idea that a particular cluster of dark pixels is the word "invoice." OCR bridges that gap, and it is the reason you can search inside scanned PDFs, copy text out of screenshots, and deposit a check by photographing it.

A very short history

OCR is much older than most people assume. Machines that could read stylized fonts existed by the mid-20th century, and banks have relied on machine-readable characters on checks (the odd-looking MICR digits along the bottom edge) since the 1950s. The landmark moment came in the 1970s, when Ray Kurzweil built a reading machine for the blind that combined the first practical omni-font OCR — software that could read ordinary typefaces rather than one special font — with a flatbed scanner and a speech synthesizer. It read printed books aloud, and its descendants power the document scanning features baked into nearly every phone today. What has changed since is not the goal but the method: hand-crafted pattern matching has largely given way to neural networks.

How OCR actually works, step by step

Modern OCR engines differ in the details, but almost all of them run the image through a similar pipeline. Understanding it explains most OCR failures you will ever encounter.

1. Binarization

The engine converts the image to pure black and white, deciding pixel by pixel what is ink and what is paper. Good engines use adaptive thresholding, which judges each region against its local background rather than one global cutoff — that is how they cope with shadows and uneven lighting. If binarization goes wrong (say, a coffee stain gets classified as ink), everything downstream inherits the damage.

2. Deskewing and cleanup

Scanned pages are almost never perfectly straight. The engine estimates the rotation angle — often by finding the orientation at which rows of text align — and rotates the image to compensate. It may also remove specks, fill small gaps in strokes, and detect page orientation (is this page upside down?).

3. Layout analysis and segmentation

Next, the engine figures out the structure of the page: which regions are text versus images or tables, where columns begin and end, and how text blocks split into lines, words, and sometimes individual characters. This is where multi-column layouts, sidebars, and tables cause trouble — segment the columns wrong and you get sentences that jump mid-line from one column to another.

4. Recognition: classic features vs. neural networks

Classic OCR recognized characters one at a time by extracting features — strokes, loops, line intersections, aspect ratios — and matching them against templates for each letter. It worked well on clean print and fell apart on anything noisy. Modern engines instead feed whole lines of text into neural networks (typically recurrent or transformer-based sequence models) that learn to map an image strip directly to a string of characters. Because they read in context, they handle touching characters, unusual fonts, and moderate noise far better than template matching ever did. Tesseract, the best-known open-source engine, made exactly this switch in version 4.

5. Language-model post-correction

Finally, the raw output is cleaned up using knowledge of the language: dictionaries, character n-gram statistics, or full language models. If the recognizer is torn between "c1ick" and "click," the language model settles it. This stage is a genuine accuracy multiplier — and also why OCR quality drops when you run an engine against a language it was not configured for.

Why handwriting and bad scans defeat OCR

Every stage of that pipeline assumes some regularity. Handwriting breaks nearly all of it: letterforms vary from writer to writer and even word to word, characters connect unpredictably, and baselines wander. Recognizing handwriting is a distinct, harder problem (often called ICR or handwritten text recognition), and even the best systems are far less reliable on it than on print. Expect cursive to fail outright with ordinary OCR tools.

Low-resolution scans fail for a more mechanical reason: information is simply gone. At 150 DPI, a lowercase letter in 10-point type may be only a dozen pixels tall — not enough to distinguish an "e" from a "c" once JPEG compression smears the edges. No algorithm can reliably recover detail that was never captured. The same applies to blurry phone photos, heavy shadows, curved pages near a book spine, and colored text on colored backgrounds that binarization flattens into mush.

Searchable PDFs vs. plain text extraction

OCR output usually takes one of two forms, and the difference matters more than people realize.

• A searchable PDF keeps the original scanned image as what you see, and adds an invisible text layer positioned behind it. The page still looks exactly like the scan, but you can now select, copy, and search the text. This is the standard output of "OCR this document" features in Acrobat and most scanner software, and it preserves the document's legal appearance — nothing visible is altered.
• Plain text extraction throws the image away and keeps only the recognized characters. You get a .txt or editable document, but layout, signatures, stamps, and formatting are gone, and every recognition error is now baked into your only copy.

For archiving, searchable PDF is almost always the right choice: mistakes in the hidden layer only hurt search, not the visible record. Extraction makes sense when you actually need to edit or process the text — a distinction closely related to why PDF and Word serve different jobs in the first place.

Tools you can actually use

• Your phone. iOS Live Text and Google Lens (or the Google Drive scanner on Android) do fast, surprisingly good OCR on photos, entirely on-device in many cases. For grabbing a paragraph or a phone number, start here.
• Tesseract. Free, open source, scriptable, supports 100+ languages, and can output searchable PDFs directly. It is the engine behind many free OCR wrappers. The command-line tool takes some setup but is excellent for batch jobs; OCRmyPDF is a popular wrapper that adds a text layer to existing PDFs using it.
• Adobe Acrobat and desktop suites. Acrobat's "Recognize Text" and commercial packages like ABBYY FineReader offer the most polished layout handling, especially for tables and multi-column documents. They cost money, but for high-volume or messy documents the accuracy difference is real.
• Cloud OCR APIs. Google Cloud Vision, Azure AI Document Intelligence, and Amazon Textract handle difficult inputs well and can parse forms and tables — with the privacy trade-off covered below.

Getting accurate results: a practical checklist

• Scan at 300 DPI. This is the long-standing sweet spot for ordinary print. Below 200 DPI accuracy drops fast; above 400 DPI you mostly gain file size, not accuracy, unless the type is tiny.
• Maximize contrast. Dark ink on light paper, even lighting, no shadows across the page. Grayscale scans usually OCR better than color for plain text.
• Keep pages flat and straight. Press books open or photograph pages individually. Deskewing fixes small tilts, not a page curving into the spine.
• Tell the engine the language. Selecting the right language (or the right combination) activates the correct dictionary and models.
• Avoid recompressing. OCR the highest-quality version you have; every JPEG round-trip destroys edge detail the recognizer needs.

Limits and the cloud privacy question

Even good OCR is probabilistic. On clean 300 DPI print, modern engines routinely exceed 99% character accuracy — which still means a typo every few sentences, so treat OCR output of contracts, medical records, or financial figures as a draft to verify, never as ground truth. Numbers deserve special suspicion: a language model can fix "teh," but it cannot know whether an invoice said 1,500 or 7,500.

Then there is the question of where the OCR runs. Free web-based "OCR my PDF" sites and cloud APIs require uploading your document to someone else's server, where it may be stored, logged, or used beyond the single conversion — the same concern that applies to online PDF tools generally. For sensitive material, prefer on-device options: your phone's built-in scanner, Tesseract or OCRmyPDF locally, or desktop software. The accuracy gap between local and cloud OCR has narrowed enough that for most documents, keeping the file on your machine costs you very little.

FAQ