Skip to content

Environment Framework

OWA's Env is the "USB-C of desktop agents" - a universal interface for native desktop automation.

Think MCP for Desktop

Just as Model Context Protocol (MCP) provides a standardized way for LLMs to connect to data sources and tools, OWA's Env provides a standardized way for agents to connect to desktop environments.

  • MCP: "USB-C of LLMs" - universal interface for AI tools
  • OWA's Env: "USB-C of desktop agents" - universal interface for native desktop automation

Quick Start

$ pip install owa

from owa.core import CALLABLES, LISTENERS
# Components available after plugin installation

Core Concepts

OWA's Environment provides three types of components for real-time agent interaction:

Direct function calls for immediate actions 

# Get current time, capture screen, click mouse
CALLABLES["std/time_ns"]()
CALLABLES["desktop/screen.capture"]()
CALLABLES["desktop/mouse.click"]("left", 2)

Event monitoring with user-defined callbacks 

# Monitor keyboard events
def on_key(event):
    print(f"Key pressed: {event.vk}")

listener = LISTENERS["desktop/keyboard"]().configure(callback=on_key)
with listener.session:
    input("Press Enter to stop...")

Background processes that can be started/stopped 

# Periodic screen capture
capture = RUNNABLES["gst/screen_capture"]().configure(fps=60)
with capture.session:
    frame = capture.grab()

Design

Unlike gymnasium.Env which uses synchronous env.step() calls, OWA's Env supports event-driven, asynchronous interactions.

  • ⚡ Real-time Performance: Optimized for responsive agent interactions (GStreamer components achieve <30ms latency)
  • 🔌 Zero-Configuration: Automatic plugin discovery via Python Entry Points
  • 🌐 Event-Driven: Asynchronous processing that mirrors real-world dynamics
  • 🧩 Extensible: Community-driven plugin ecosystem

Quick Navigation

Section Description
Environment Guide Complete system overview and usage examples
Custom Plugins Create your own environment extensions
CLI Tools Plugin management and exploration commands

Built-in Plugins:

Plugin Description Key Features
Standard Core utilities Time functions, periodic tasks
Desktop Desktop automation Mouse/keyboard control, window management
GStreamer Hardware-accelerated capture Fast screen recording