Interactive material · Computer vision, part 1
How a Computer Vision Model
Processes a Mammogram
A screening mammogram carries clinical meaning to a trained reader. A computer vision model receives numerical inputs. This page follows the path from displayed image to number grid to model output, using a synthetic image you can zoom into, change, and test.
No math required. Nothing here is a real medical image, a real patient, or a real diagnostic system.
What you will be able to explain
- How a displayed image becomes numerical input for a computer vision model.
- How classification, detection, segmentation, and response maps differ.
- Why a model score, a probability, and a clinical diagnosis have different meanings.
- Which evidence and monitoring questions belong in a real clinical AI review.
Choose the depth you need
Essentials, about 8 minutes. Follow the five links below, then complete the quick check.
Full walkthrough, about 15 minutes. Work through every section and try each control.
1 The human view
What a person sees
A trained reader looks at this and sees anatomy: tissue, a chest wall on the left, an outline, and possibly something that stands out from its surroundings. Meaning arrives all at once.
Synthetic teaching image · not a real mammogram
Drawn by code from a fixed random seed. Every learner sees exactly the same image.
Two different acts of seeing
A person sees a breast, and inside it maybe a suspicious area. Those are human concepts, built from years of training and from knowing what a body is.
A model receives numerical inputs rather than a patient's story. During training, it can learn internal patterns associated with labels such as "breast" or "tumor." Those learned associations differ from human clinical understanding and can fail when a new image differs from the training data.
On this page, the input is a rectangle of numbers describing the brightness of each tiny square.
How this differs from a real examination
- A routine bilateral screening mammogram normally includes four standard views: craniocaudal (CC) and mediolateral oblique (MLO) views of each breast. Digital breast tomosynthesis can reconstruct many image slices from its acquisitions.
- Readers compare left with right, and this year with last year. A single flat picture, as used here, removes most of that context.
- Clinical mammography images often preserve more tonal detail than this page's 8-bit example. Dedicated presentation processing controls how those values appear on screen.
- This page uses one simplified synthetic image so that the pixel-level journey stays visible. Treat it as a diagram, not as a specimen.
2 The pixel view
Zoom in far enough and the picture disappears
Move the yellow box, then zoom. At full magnification there is no longer an image at all, only squares of brightness, each one a single number.
Click or tap the image to move the box. With the box focused, use the arrow keys; hold Shift for larger steps.
Three steps: whole image, a 32 by 32 patch, then an 8 by 8 patch with the numbers shown.
Magnified region
A grid of 64 pixels. Use the arrow keys to move between them, Enter or Space to pin one. Each number is that pixel's brightness from 0, black, to 255, white.
Stored value, displayed brightness, and model input
On this page, values from 0 to 255 directly control brightness. In clinical imaging, the value stored in a file, the brightness shown to a reader, and the number sent to a model can differ. Display instructions and preprocessing help determine each representation.
Two honest simplifications on this page:
- Many digital mammography and diagnostic x-ray images preserve more tonal levels than this 8-bit teaching image. Twelve meaningful bits inside a 16-bit container is one possible arrangement. The details vary by device and image object.
- Model pipelines commonly preprocess images by cropping, resizing, rescaling, or filtering them. The order and exact steps vary by system.
3 The numerical representation
From picture to array
Here is the patch you selected, written out as numbers. In this simulation, a grid like this repeated across the whole image is the complete input. Real systems may also use multiple views, prior examinations, or metadata.
Raw brightness values, straight from the image.
The whole journey, in four steps
Resizing in this simulation
Full image
256 × 256
Model input
64 × 64 = 4,096 numbers
The simulation on this page shrinks the image to 64 by 64 before analyzing it, which is why the version on the right is what actually gets scored. Real systems use larger inputs, and choosing that size is a genuine design trade-off between fine detail and computation.
4 A rule-based pattern ladder
How this teaching simulation builds a score
This page applies fixed comparisons written by its author. The sequence offers simplified intuition for one convolution-style workflow: local differences become edges, focal responses, coarse regions, and finally a score. Step through the teaching simulation below.
Simulation step 1
Pixel differences
Is this number bigger or smaller than its neighbor? Nothing more.
Simulation step 2
Edges and textures
Differences that line up become boundaries and grain.
Simulation step 3
Shapes and densities
Compact areas that stand out from what surrounds them.
Simulation step 4
Coarse regions of response
The strongest response in each region is recorded for this display.
Simulation step 5 · a single number comes out
The simulation combines four measured features: the strongest focal response, how concentrated that response is, how sharp it is, and how much grain is present. Fixed rule-based calculations turn those measurements into one output. A trained model learns its weights from examples, which is the subject of the next lesson.
5 Three tasks, plus a response map
Classification, detection, segmentation
Classification, detection, and segmentation answer different technical questions. The fourth tab shows the internal filter response from this teaching simulation. A predictive localization map, an internal response map, and a post hoc attribution map each represent a different quantity.
Simulated output · synthetic image
Classification
| Concept | Meaning | What comes out |
|---|---|---|
| Image classification | Assigns categories or scores to a defined unit, such as an image, breast, or examination | One or more labels or scores |
| Object detection | Identifies candidate locations of possible findings | Zero or more boxes, points, or markers, often with per-candidate scores |
| Segmentation | Estimates the boundaries of a finding | Per-pixel scores that are commonly converted into an outline or mask |
| Predictive localization map | Produces location-specific scores as a trained model output | A map of location-specific or per-pixel scores |
| Internal response map | Shows a defined internal quantity, such as this simulation's filter response | A shaded overlay tied to a specific calculation |
| Attribution or saliency map | Estimates which parts of an input influenced a particular output | An explanatory overlay that requires method-specific validation |
6 The model output
A score, a probability, and a diagnosis have different meanings
Everything so far produces one number. The model's documentation and validation determine what that number means.
Simulated suspicion score
—
—
A threshold turns a continuous score into a yes or no. Move it and watch the same score cross the flagging threshold.
What this number is
- The output of this simulation on this synthetic image, produced by fixed filters written for teaching.
- A relative measure. Higher means the image contains more of what these filters respond to.
- A quantity that a threshold can convert into an action, such as flagging an examination for a closer look.
Limits of this number
- Cancer can be present after a low score and absent after a high score.
- A score of 0.72 remains a unitless score. It represents a 72% estimated probability only when the model has been designed, evaluated, and shown to support that interpretation in the intended population.
- The radiologist and the surrounding clinical workflow determine what happens next.
Four terms that make threshold trade-offs precise
- Sensitivity is the share of cases with the condition that the system flags.
- Specificity is the share of cases without the condition that the system leaves unflagged.
- Positive predictive value is the share of flagged cases that have the condition.
- Negative predictive value is the share of unflagged cases that do not have the condition.
The threshold changes sensitivity and specificity. Predictive values also depend on how common the condition is in the population being screened. Every metric needs a defined population, reference standard, threshold, and measure of uncertainty.
Watch one threshold affect a group
This teaching cohort contains 100 invented examinations: 20 where the condition is present and 80 where it is absent. The scores demonstrate the math and represent no clinical system, screening rate, or expected performance.
| Reference standard | Flagged | Below threshold | Total |
|---|---|---|---|
| Condition present | … true positives | … false negatives | 20 |
| Condition absent | … false positives | … true negatives | 80 |
7 Change the image, watch the score
Image conditions change what the model receives
The first five controls change the image conditions while the drawn finding stays fixed. The final control changes how clearly that finding appears against its surroundings. Some changes barely move the score. Others move it a lot. That gap is one of the most important things to understand about computer vision.
Teaching simulation · not a validated model
Score now
—
Change from baseline
—
Adjustments
A schematic global shift. Real exposure and detector differences affect images in more complex ways.
A schematic grain control. Real noise depends on dose, detectors, processing, acquisition, and transmission.
A schematic proxy. Real positioning changes which anatomy is captured and how tissue is projected and compressed.
An artifact, a marker, or a region simply not captured.
Changes the contrast of the drawn finding against the surrounding image. This control is included for comparison with the image-condition changes above.
Why this matters outside a teaching page
A model is fitted to the images it was shown. When the images it meets later differ, performance can differ too, even when the anatomy is identical. Common sources of that difference include:
- Different scanner makes and models
- Image quality and dose
- Patient positioning
- Preprocessing choices
- Artifacts and implants
- Missing or incomplete views
- Differences between hospitals and populations
- Shortcuts tied to labels, markers, borders, scanners, sites, or processing
Watching this over time, deciding when a difference matters, and knowing what to do about it are their own subjects. Later lessons cover monitoring and drift.
8 Before you go
The seven things worth carrying away
- Screening, diagnostic workup, and diagnosis serve different purposes. Screening mammography looks for signs in people without symptoms. A diagnostic mammogram or other workup investigates a symptom or screening finding, often with additional views. A definitive diagnosis may require further clinical evaluation and, when indicated, tissue sampling.
- A suspicious result does not mean cancer is present. In U.S. screening programs, most examinations recalled for additional work-up do not result in a cancer diagnosis. Rates vary by age, risk, screening interval, and program.
- A non-suspicious result does not guarantee cancer is absent. Findings can be missed, by people and by models.
- A response map needs a precise name and evidence. Saliency, activation, attention, lesion-likelihood, and filter-response maps represent different quantities. A convincing overlay can still give a poor account of model behavior.
- The score on this page is illustrative. It came from rule-based filters over a drawn image, not from a trained system.
- Evidence must match the intended use. Look for representative data, independent and multisite testing, relevant subgroup results, local workflow evaluation, prospective evaluation when feasible, applicable regulatory review, planned human-device interaction, and ongoing monitoring.
- Nothing here supports a medical decision. If you have a question about your own screening, speak to a clinician.
Evidence before use
- What is the intended use, input, output, user, and place in the workflow?
- How was the reference standard, the method used to determine the correct answer, established? Who assigned labels, what follow-up was available, and how were uncertain or disputed cases handled?
- Were training, tuning, and test partitions separate at the patient level or higher? What controls kept information from test cases out of model development, a problem called data leakage?
- Was the completed model tested on external or otherwise development-independent data spanning representative sites, scanners, protocols, populations, and subgroups relevant to the planned use?
- Are statistical uncertainty and subgroup results reported alongside overall performance? When outputs are interpreted as probabilities, is calibration reported?
- How does the human-AI team perform compared with the current clinical workflow?
Controls after launch
- Who verifies inputs and outputs, who can override the system, and where are disagreements or unexpected results reported?
- Who owns monitoring, which metrics and subgroups are reviewed, and how often does review occur?
- How will changes in equipment, software, workflow, or patient population trigger reassessment?
- What thresholds trigger investigation, temporary pause, revalidation, or retirement, and who has authority to act?
Regulatory note. The FDA lifecycle source linked below remains January 2025 draft guidance. FDA labels it "Not for Implementation" and states that it contains nonbinding recommendations.
Quick check
A vendor shows a score and a heatmap
A vendor says one screening examination received a score of 0.78 and the heatmap shows where the cancer is. Which response best applies the ideas from this lesson?
Choose a response to see feedback.
Sources and technical notes
- American College of Radiology. ACR Practice Parameter for the Performance of Screening and Diagnostic Mammography. Revised 2023. States that the screening examination should ideally be limited to technically adequate bilateral craniocaudal and mediolateral oblique views. Supports the note in section 1 that a real examination normally includes four images.
- DICOM PS3.3, Image Pixel Module (C.7.6.3). Defines Pixel Data, Bits Allocated, Bits Stored, and Photometric Interpretation, including arrangements beyond the 8-bit values used by this teaching page. Supports the section 2 distinction between stored values and displayed brightness.
- Tejani AS, Klontzas ME, Gatti AA, et al. Checklist for Artificial Intelligence in Medical Imaging (CLAIM): 2024 Update. Radiology: Artificial Intelligence. 2024. Covers acquisition, preprocessing, intended use, internal and external testing, and prospective evidence. Supports the validation expectations in section 8.
- Developing and evaluating a clinical prediction model: a step-by-step guide. BMJ. 2024;386:e078276. Discusses discrimination, calibration, external validation, and transportability. Supports the section 6 point that a score is only a probability if the model was calibrated and validated to be one.
- Arun N, Gaw N, Singh P, et al. Assessing the Trustworthiness of Saliency Maps for Localizing Abnormalities in Medical Imaging. Radiology: Artificial Intelligence. 2021;3(6):e200267. Evaluates saliency maps on localization utility, sensitivity to model weights, repeatability, and reproducibility, and recommends detection or segmentation models where localization is the goal. Supports the heatmap caution in sections 5 and 8.
- US Food and Drug Administration. Artificial Intelligence-Enabled Device Software Functions: Lifecycle Management and Marketing Submission Recommendations. Draft guidance, January 2025. Addresses intended-use populations, multisite testing, human-device interaction, bias, workflow, and lifecycle performance monitoring. Supports the acquisition-variation list in section 7 and the oversight point in section 8.
- National Cancer Institute. Breast Cancer Screening (PDQ), Health Professional Version. Covers false positives with additional testing, false negatives with delayed diagnosis, and downstream evaluation including biopsy. Supports the first three cautions in section 8.
Next material
Where the weights come from
This page took the model's behavior as given. Every number that turned a pattern into a score was fixed in advance. In a real system those numbers are learned, and something has to teach them what a correct answer looks like.
How does a computer vision model learn which image patterns are associated with cancer, and who determines the correct answer?