SeeClick

Open-source visual GUI agent that locates interface elements from screenshots.

Open Source Automation & Agents
Visit Website ↗ by Nanjing University

What is SeeClick?

SeeClick is an open-source visual GUI agent from Nanjing University and Shanghai AI Lab that operates interfaces purely from screenshots. Built on a vision-language model with a dedicated GUI-grounding pretraining strategy, it can locate text, widgets, and icons across mobile, desktop, and web environments. Its ScreenSpot benchmark became the standard test for GUI grounding accuracy.

Key features

  • GUI grounding pretraining for precise element location
  • Screenshot-only interaction — no HTML or accessibility trees needed
  • Cross-platform: iOS, Android, macOS, Windows, web
  • ScreenSpot grounding benchmark
  • Open model and training code

Pros

  • Research-backed with a published ACL 2024 paper
  • Foundational for screenshot-based computer-use agents
  • Open weights and code

Cons

  • Research codebase, not a polished tool
  • Large model — needs serious GPU
  • Limited recent development activity

SeeClick at a glance

PricingOpen Source
CategoryAutomation & Agents
Platformslinux, api
CompanyNanjing University
Official websitegithub.com
gui-agentgroundingvisionresearchcomputer-use

How to choose the right Automation & Agents tool

Automation and agent tools promise to do work for you, so judge them on reliability before cleverness. Map the exact workflow you want to automate, then check the tool's integrations cover every app in that chain — one missing connector breaks the whole flow. Look for clear triggers, sensible limits, and logs you can read when something goes wrong, because something eventually will. For agents that act on your behalf, approval controls matter: you should be able to review messages, purchases or changes before they happen, especially at the start. Pricing usually scales with runs or tasks, so estimate your monthly volume honestly. Start with one low-risk workflow, measure the time it truly saves, and expand from there.

Compare all Automation & Agents tools →

SeeClick FAQs

What is SeeClick?

SeeClick is an open-source visual GUI agent from Nanjing University and Shanghai AI Lab that operates interfaces purely from screenshots. Built on a vision-language model with a dedicated GUI-grounding pretraining strategy, it can locate text, widgets, and icons across mobile, desktop, and web environments. Its ScreenSpot benchmark became the standard test for GUI grounding accuracy.

What are the main features of SeeClick?

SeeClick offers GUI grounding pretraining for precise element location, Screenshot-only interaction — no HTML or accessibility trees needed, Cross-platform: iOS, Android, macOS, Windows, web, ScreenSpot grounding benchmark and Open model and training code.

How much does SeeClick cost?

SeeClick is open source, so the software itself is free to download, use and self-host. Your only costs are the infrastructure you run it on and any paid services you connect to it.

What are the pros and cons of SeeClick?

Strengths of SeeClick include Research-backed with a published ACL 2024 paper; Foundational for screenshot-based computer-use agents; Open weights and code. On the downside, Research codebase, not a polished tool; Large model — needs serious GPU; Limited recent development activity.