12 · Surgical Image Captioning
Image captioning expresses a surgical scene in natural language, describing visible instruments, tissue and their relationships.
Input and output: A surgical image → a concise scene description.
Source data and labels: A frame and instrument–action–target triplets. This example has “grasper–grasp–gallbladder” and “hook–dissect–cystic artery” labels.
Construction method: The paper uses Qwen3-VL-235B to enrich incomplete descriptions, forming image–caption pairs. The diagram shows source labels separately from the enrichment step; the exact model prompt remains to be supplied.
Figure 2. Structured triplets and natural-language captions are different forms of supervision.
Q: Describe the following surgical image.
A: A grasper entering from the left holds tissue, while a hook entering from the upper right contacts exposed tissue near the center. The liver surface is visible in the background.