Build with Perception Agents — Emile Baizel & Shruti Arora, Amazon AGI Lab

A coding agent says it added a duck icon to a podcast page; a browser verification pass checks whether the change actually appeared. That audience demo captures the loop Emile Baizel and Shruti Arora build around two perception agent primitives. Visual annotation lets a person select an element, draw on the page or drop a pin, then export structured feedback for a coding agent. Visual verification checks the rendered site against design expectations and user flows. The annotation step is deterministic and produces JSON; the verification step gives the agent a way to inspect the result of its work. Arora starts from a podcast landing page whose heading, colors and buttons need changes, then connects the browser extension to a coding agent through a local bridge. Participants modify the extension's permissions and add an Apply Changes button, turning exported feedback into a workflow that edits the code, runs verification and produces a report. Easy and medium paths let them follow the wiring directly or solve more of the integration themselves. A guest demonstration uses a simulated Bee conversation as another source of design feedback, feeding the same edit and verification loop. Audience examples extend the idea to spoken feature discussions, project planning and recorded UI feedback, with several integrations still at the concept stage. The closing discussion keeps the focus on making collaboration visual and checking the rendered result, rather than accepting a coding agent's claim that the change is done. Speaker info: - https://www.linkedin.com/in/emilebaizel - https://www.linkedin.com/in/shruti-arora-0730 - https://github.com/amazon-agi-labs/perception-agents-workshop Timestamps: 0:00 - Meet Emile Baizel and Shruti Arora 1:50 - Collaboration, perception and reliability 3:40 - Browser workflows with Amazon Nova Act 5:43 - Understanding screens beyond clicking 7:43 - Deterministic visual annotation 8:25 - Verify designs and user flows 9:23 - The frontend feedback scenario 12:52 - Draw, select an element or drop a pin 13:46 - Generate design specs and verification reports 15:26 - Wire the two primitives together 18:29 - Find the workshop repository 19:26 - The…