Overview
This project won 1st place at an internal LGE hackathon. The agent automatically detects UX guideline violations in in-progress application screens by combining RAG-retrieved guideline knowledge with VLM-based visual analysis.
Problem
Design teams maintain detailed UX guidelines — typography rules, spacing requirements, accessibility standards, component usage specs. Validating screens against these guidelines manually is time-consuming and inconsistent, especially as products evolve and guidelines are updated.
Approach
The agent runs as a LangGraph workflow with three main stages:
Guideline retrieval — given a screenshot and an optional focus area (e.g., "check the navigation bar"), use RAG to retrieve the most relevant sections of the UX guideline document.
Visual analysis — pass the screenshot and retrieved guidelines to a VLM. Ask it to describe what it observes in the screen relevant to each guideline section.
Violation detection — a reasoning step compares the VLM's visual description against each guideline rule and determines whether a violation exists, generating a structured report with locations and explanations.
LangGraph handles the orchestration: conditional branching based on screen type, retry logic when the VLM output is ambiguous, parallel processing of multiple guideline categories.
Why LangGraph
The workflow needed more than a simple chain. Different screen types require different guideline categories. Some steps need to run in parallel (checking typography and spacing simultaneously). And some VLM outputs need a verification pass before being included in the final report.
LangGraph's explicit state machine made it straightforward to express these control flows without ad hoc conditional logic.
Results
Won 1st place at the LGE internal hackathon. The agent successfully detected real violations in sample screens across typography, component misuse, and accessibility categories. The structured report output made it actionable for design reviews without requiring a developer to interpret raw model output.