13 · Surgical Video Captioning
Video captioning describes actions, instrument movement and changes in their temporal order, extending scene understanding beyond a single image.
Input and output: A surgical video clip → a description of its main action and temporal changes.
Source data and labels: Video with time-aligned action labels. In the example, “irrigator–aspirate–fluid” occurs at each of the four illustrated moments; the caption additionally describes movement and withdrawal.
Construction method: Select a clip and align its existing action labels along its timeline. The paper's Qwen3-VL-235B pipeline enriches incomplete descriptions. Pair the clip with a caption covering the action and its progression.
Figure 3. Four moments from one 23-second clip. Times are relative to the start of the video.
Q: What is happening in this surgical video?
Input video: Watch the example video.
A: An irrigator entering from the right moves its tip over a blood-stained tissue surface and aspirates fluid. It is withdrawn from view near the end of the clip.