Own desktop
Give the agent a desktop of its own.
You want your agent to use a graphical program without touching your screen or seeing your other windows.
Capturedmcp-xsession 0.5.5
YouCreate a desktop for this task and show me that it works.
Tool callxsession_create { "name": "guide-demo", "resolution": "1280x720", "wm": "openbox", "audio": "off" }
What happensState running, with the display server, remote desktop service and window manager started; RDP on 127.0.0.1 port 33890 over TLS.
Tool callxsession_get_env { "sessionId": "guide-demo" }
What happensDISPLAY :101 and the session's XAUTHORITY path; WAYLAND_DISPLAY is null because there is no Wayland compositor.
Tool callrdp_connection_create { "connectionId": "guide-demo", "host": "127.0.0.1", "port": 33890, "username": "xsession", "password": "<token>", "security": "tls", "graphicsMode": "rfx" }
What happensThe viewer connects.
Tool callrdp_screenshot { "connectionId": "guide-demo", "filename": "desktop.png" }
AgentThe desktop is up and answering. I saved desktop.png.
No screen
Run graphical tests where there is no screen.
Your agent runs on a server or in a container, with no display and no graphics card, and it has to check something that only works in a real graphical program.
Illustrative conversation
YouThere is no screen in this pod. Start a desktop, open the app and tell me what it shows.
What happensA shell command shows that DISPLAY is empty: no X11 session and no graphics card are available in this pod.
Tool callxsession_create { "name": "headless-ui", "resolution": "1280x720", "wm": "openbox", "audio": "off" }
What happensThe session starts in state running, with its own display server and an RDP endpoint on loopback.
Tool callxsession_get_env { "sessionId": "headless-ui" }
What happensReturns DISPLAY and XAUTHORITY for the new desktop, and the rdp block.
What happensA shell command starts the app with those two values set.
Tool callrdp_connection_create { "connectionId": "headless-ui", "host": "127.0.0.1", "port": "<port from the rdp block>", "username": "xsession", "password": "<token>", "security": "tls", "graphicsMode": "rfx" }
Tool callrdp_screenshot { "connectionId": "headless-ui", "filename": "app.png" }
AgentThere was no display in this pod, so I created one. The app opened and its main window is showing; I saved app.png.
Run an app
Open an app on it and watch.
You want a graphical program running somewhere it cannot disturb you, with a picture of what it shows.
Capturedmcp-xsession 0.5.5
YouOpen the browser on that desktop, load our docs page and show me.
Tool callxsession_get_env { "sessionId": "guide-demo" }
What happensReturns DISPLAY :101 and the XAUTHORITY path inside the session's runtime directory.
What happensA shell command starts the browser with DISPLAY and XAUTHORITY set to those values and the docs URL as its page.
Tool callrdp_screenshot { "connectionId": "guide-demo", "filename": "desktop.png", "delay": 1500 }
What happensA 1280 by 720 image is published as a resource: the browser window on the Openbox desktop.
AgentThe browser is open on the documentation page inside the session. I saved desktop.png.
The captured image shows the desktop with the browser window open on the documentation page.
Cross-browser
Check a page in two browsers and at two sizes.
You changed a web page and want to see how it looks in more than one browser, and what happens when the window gets smaller, without opening anything on your own screen.
Capturedmcp-xsession 0.5.5 · mcp-rdp 0.9.4
YouOpen the docs page in Firefox and Chromium side by side, close the notices, then show me the page at a narrower desktop size.
Tool callxsession_create { "name": "crossbrowser", "resolution": "1280x720", "wm": "openbox", "audio": "off" }
What happensState running, with the display server, remote desktop service and window manager started.
Tool callxsession_get_env { "sessionId": "crossbrowser" }
What happensReturns DISPLAY, XAUTHORITY and the rdp block.
Tool callrdp_connection_create { "connectionId": "crossbrowser", "host": "127.0.0.1", "port": "<port from the rdp block>", "username": "xsession", "password": "<token>", "security": "tls", "graphicsMode": "rfx" }
Tool callrdp_start_video { "connectionId": "crossbrowser", "audio": false, "filename": "xs-crossbrowser.webm", "condense": { "profile": "balanced", "keepOriginal": false } }
What happensReturns status recording and the condense settings.
What happensShell commands start Firefox, then Chromium at the right half of the screen, each on the docs page with the session's DISPLAY and XAUTHORITY set.
Tool callrdp_pointer { "connectionId": "crossbrowser", "action": "click", "x": 599, "y": 178 }
What happensFour clicks like this close the notice and decline the optional diagnostics in each browser.
Tool callrdp_pointer { "connectionId": "crossbrowser", "action": "scroll", "x": 320, "y": 420, "direction": "down", "clicks": 12 }
Tool callrdp_screenshot { "connectionId": "crossbrowser", "filename": "side-by-side.png" }
Tool callrdp_pointer { "connectionId": "crossbrowser", "action": "click", "x": 1228, "y": 20 }
What happensThe Chromium window is maximised and shows the wide layout of the page.
Tool callxsession_action { "sessionId": "crossbrowser", "action": "resize", "width": 800, "height": 600 }
What happensReturns status ok and the new resolution, 800 by 600.
Tool callrdp_pointer { "connectionId": "crossbrowser", "action": "move", "x": 400, "y": 300 }
Tool callrdp_screenshot { "connectionId": "crossbrowser", "filename": "narrow.png", "delay": 1800 }
Tool callrdp_stop_video { "connectionId": "crossbrowser" }
What happensReturns the condensed video: about 27 seconds kept from about 110, 7 kept ranges and 6 settled events.
AgentBoth browsers show the page. At 800 by 600 the page drops its side panel and puts the section menu in a bar at the top. The screenshots and the video are saved.
Resize
Change the desktop size while it runs.
You want to see how something looks at another size, or the first size was a guess.
Capturedmcp-xsession 0.5.5
YouMake the desktop 1600 by 900.
Tool callxsession_action { "sessionId": "guide-demo", "action": "resize", "width": 1600, "height": 900 }
What happensReturns status ok and the new resolution, 1600 by 900.
Tool callxsession_list {}
What happensThe session record now reports resolution 1600 by 900, and its state is still running.
AgentThe desktop is now 1600 by 900. A connected viewer follows without reconnecting.
Freeze
Freeze the desktop, then carry on.
You want the agent to stop working for a while without losing what is open, then continue exactly where it was.
Capturedmcp-xsession 0.5.5
YouPause that desktop, then resume it.
Tool callxsession_action { "actions": [ { "sessionId": "guide-demo", "action": "stop" }, { "sessionId": "guide-demo", "action": "resume" } ] }
What happensTwo results, both successful; the summary reads total 2, succeeded 2, failed 0.
AgentThe desktop was frozen and thawed. Nothing was closed. Resume before destroying a stopped session: destroying a frozen one is slow and can time out.
Recover
Find a session again after a crash.
The server stopped without a clean shutdown, or a colleague released a desktop, and you want to finish it properly instead of hunting for processes.
Illustrative conversation
YouList the sessions nobody owns and take over the right one.
Tool callxsession_list { "includeOrphaned": true }
What happensThe orphaned list holds one summary: id, name demo, state running, audio and creation time. It carries no ports or paths on purpose.
Tool callxsession_adopt { "sessionId": "<exact id from the list>" }
What happensReturns the full session record: components, RDP port, runtime directory.
AgentI now own the demo session. It was still running. Should I keep it or destroy it?
YouDestroy it.
Tool callxsession_destroy { "sessionId": "demo" }
What happensThe session's processes stop and its files are removed.
Record
Record the desktop with sound and picture.
You want proof of what an application did on its desktop, with the sound it played, so a person can watch and listen to it afterwards. Or you are testing an application that listens to a microphone.
Illustrative conversation
YouRecord a one-minute video of the desktop with sound while the app runs, then show me whether it made any sound.
Tool callxsession_create { "name": "av-check", "resolution": "1280x720", "wm": "openbox", "audio": "bridge" }
What happensThe session starts with an audio server among its components.
Tool callxsession_get_env { "sessionId": "av-check" }
What happensReturns DISPLAY, XAUTHORITY, a PULSE_SERVER path and the rdp block.
Tool callrdp_connection_create { "connectionId": "av-check", "host": "127.0.0.1", "port": "<port from the rdp block>", "username": "xsession", "password": "<token>", "security": "tls", "graphicsMode": "rfx" }
Tool callrdp_start_video { "connectionId": "av-check", "audio": true, "maxDurationSecs": 60, "filename": "av-check.webm" }
What happensReturns status recording, with the dimensions and the frame rate.
What happensA shell command starts the app on that DISPLAY, with PULSE_SERVER set, and lets it run.
Tool callrdp_stop_video { "connectionId": "av-check" }
What happensReturns the video resource and its statistics: duration, frames, hasAudio true and a non-zero count of audio packets.
AgentI recorded 60 seconds of the desktop with sound. The video carries an audio track with packets, so the app did play sound; I saved av-check.webm.
To test a microphone instead, ask the agent to play a prepared audio file into the desktop and report how much was sent. See the MCP RDP tools for the recording and sound options.