M1–M4 done · open source

The AI assistant for Linux that never hides what leaves your machine.

Ask anything about your Linux system and get answers grounded in real data — while seeing exactly what, if anything, leaves your machine. Veya is an open-source, local-first AI assistant for Ubuntu/Linux: an Apple Intelligence alternative that runs on local models first and logs every cloud call.

View on GitHub How it works

you › anything urgent in my notifications?

veya › Yes — a failing CI run on veya#214 and a 3pm
        meeting that moved to 4. The rest can wait.
       ↳ Mistral (cloud) · 0.9 KB sent · logged to audit

See it in action

A global overlay — summon it anywhere, ask anything about your system, get an answer backed by real data.

Veya overlay appearing on a Linux desktop with an empty search bar — 1 — Invoke Veya anywhere with a keyboard shortcut

Question typed into the overlay: 'What takes the most RAM on my PC?' — response loading — 2 — Ask anything about your machine

Veya answer showing top 10 processes by RAM usage with a summary — 3 — Get a grounded answer drawn from live system data

Built around three commitments

🔒

Local-first privacy

Local models are first-class. Every source of context — clipboard, files, notifications, screen — is gated by per-source permissions, default-deny.

👁️

Transparency

An audit log records every tool invocation and every cloud request — backend, model, token counts, never the content. Cloud usage is always user-visible, surfaced live over D-Bus.

🧩

System-wide, not app-bound

One daemon, one D-Bus contract (org.veya.Veya1), any number of frontends — GTK4 overlay window, GNOME Shell extension (keyboard summon, mic button), CLI, or your own client.

Architecture at a glance

A central daemon owns all intelligence. Frontends are thin clients over D-Bus; system access is isolated in an MCP server behind a safety layer.

---
config:
  theme: base
  themeVariables:
    background: '#0b0e14'
    mainBkg: '#151a26'
    primaryColor: '#151a26'
    primaryBorderColor: '#232a3a'
    primaryTextColor: '#e6e9f0'
    lineColor: '#6ea8fe'
    edgeLabelBackground: '#0b0e14'
    clusterBkg: '#11151f'
    clusterBorder: '#232a3a'
    titleColor: '#e6e9f0'
    fontFamily: 'ui-sans-serif, system-ui, sans-serif'
    fontSize: '13px'
---
flowchart TD
    subgraph FE[Frontends]
        UI["Overlay UI\nGTK4 / Gir.Core"]
        GS["GNOME Shell ext\nGJS · keyboard summon · mic"]
        CLI[CLI]
    end

    subgraph D["Daemon — Veya.Daemon"]
        DCore["sessions · context\nper-source permissions · audit log"]
        Router["model router · FallbackInferenceBackend"]
        Ctx["personal context index\nSQLite + sqlite-vec · local embeddings"]
        Notif["notification intelligence\ncapture · digest · answer"]
        Voice["voice I/O · AskVoice\nWhisper STT · espeak-ng TTS · local"]
        DCore --> Router
        DCore --- Ctx
        DCore --- Notif
        DCore --- Voice
    end

    FE -->|"D-Bus · org.veya.Veya1"| D

    Router --> Ollama["Ollama\nlocal · no cloud egress"]:::local
    Router --> Mistral["Mistral (default)\ncloud · user-visible"]:::cloud
    Router --> Claude["Claude (optional)\ncloud · user-visible"]:::cloud

    D -->|"MCP over stdio"| MCP["McpServer — Veya.McpServer\nread-only: system info · processes · memory/disk · journald · APT · systemd\nwrite (permission-gated): set_clipboard · read_screen_text\nsafety layer: allowlist · timeouts · output caps · audit log"]

    MCP --> OS["Ubuntu system\nunprivileged · polkit later"]

    classDef local fill:#0d1a12,stroke:#4ade80,color:#4ade80
    classDef cloud fill:#130d1f,stroke:#b388ff,color:#b388ff

Daemon

Long-running user service. Owns sessions, context, permissions, the audit log, and the model router. Local-first FallbackInferenceBackend: try Ollama, fall back to the configured cloud backend (Mistral by default, Claude optional). Folds personal context and notification digest into every answer.

McpServer

System-action layer over the MCP protocol. Read-only tools: system info, processes, memory/disk, journald, APT, systemd. Write tools: set_clipboard and read_screen_text (screen awareness) — both permission-gated behind the central safety layer.

Personal context + notifications

Personal context index grounds answers in user-approved files via local embeddings + SQLite/sqlite-vec. Notification intelligence captures and digests desktop notifications. Both permission-checked, both degrade gracefully when denied.

Safety layer

Every shell command goes through an allowlist, timeouts, output caps, and the audit log. Daemon and MCP server run unprivileged as user services.

Roadmap

Milestones 1–4 are done. Everything builds on the same daemon + D-Bus + MCP spine.

M1

MVP — end-to-end answer on screen

Daemon skeleton, D-Bus Ask, MCP server with read-only system tools, Claude API backend, minimal overlay window.
M2

Local models + first write actions

Ollama local backend; config-selectable cloud tier — Mistral (default) or Claude; local-first FallbackInferenceBackend; permission-gated clipboard write.
M3

Personal context + notification intelligence

Embeddings index (SQLite/sqlite-vec) over user-approved files; notification capture and digest — summarize, prioritize, answer. Both permission-gated and audit-logged.
M4

Voice, screen awareness, GNOME polish

Screen awareness (read_screen_text via XDG portal + Tesseract) ✓ — GNOME Shell extension (keyboard summon, cloud badge, mic button) ✓ — Voice I/O (local Whisper STT + espeak-ng TTS via AskVoice) ✓.

Next: packaging (Flatpak + .deb) so anyone can install Veya, a second frontend beyond GNOME to prove the D-Bus contract, and privileged write actions through polkit (ADR-0003) — acting on the system, not just reading it, without ever assuming root.

Open source. AGPL-3.0. Contributions welcome.

One issue per branch; every PR runs the canonical build + test + format check in CI. Contributions are accepted under the same license and require a Developer Certificate of Origin sign-off.

git clone https://github.com/s3ba-b/veya.git

Build & run instructions live in the repo’s README.

Star it on GitHub