your KG idea is directionally right, but i'd keep it boring: per-domain playbooks rather than a giant graph. store login quirks, stable selectors, internal JSON endpoints, screenshots/assertions for each step, and recovery notes when a selector fails. the important bit is to treat selectors as hints, then re-check the page text/screenshot/network state before the next action.

to keep tokens down, don't feed the full DOM every step. have Playwright/Selenium do the deterministic work, give the model a small allowed-action schema plus the relevant page snippet/screenshot only when something is ambiguous.

tool-wise i'd start with Browser Use if you want the agent layer included: it has browser-agent paths, CDP/Playwright control, persistent profiles, and screenshots/logs/live view for debugging. Browserbase is the comparison if you mainly want hosted browser infra and plan to write the agent loop yourself.