Input Control (Keyboard, Mouse)

What We’re Testing

Input from the browser viewer is carried over a dedicated WebRTC DataChannel named "input". This channel is created by the viewer (browser-creates, ordered: true) so keystrokes always arrive in sequence. The "video" channel and "input" channel are independent SCTP streams — video drops are transparent to keyboard delivery.

Browser side (RemoteDesktopViewer.tsx):

The canvas element captures all mouse and keyboard DOM events via React event handlers:

• onMouseMove → sends { type: "mousemove", x, y } where x/y are scaled from canvas coordinates to host pixel coordinates via scaleCoordinates()
• onMouseDown / onMouseUp → sends { type: "mousedown"/"mouseup", x, y, button: "left"/"right"/"middle" }
• onWheel → sends { type: "scroll", deltaX, deltaY } (raw pixel delta from WheelEvent)
• onKeyDown / onKeyUp → sends { type: "keydown"/"keyup", key, code, modifiers: ["ctrl","shift","alt","meta"] }
• onContextMenu → e.preventDefault() to suppress the browser context menu on right-click

The canvas has tabIndex={0} so it receives keyboard focus. All events call e.preventDefault() to block browser defaults (e.g., browser scrolling, tab focus changes).

Coordinate scaling uses scaleCoordinates():

scaleX = hostResolution.width  / canvas.getBoundingClientRect().width
scaleY = hostResolution.height / canvas.getBoundingClientRect().height
x = Math.round((clientX - rect.left) * scaleX)
y = Math.round((clientY - rect.top)  * scaleY)

Agent side (input_windows.go): The windowsInputInjector calls Win32 SendInput via syscall. The inputUnion struct is carefully sized to 40 bytes to match the Win32 INPUT struct on 64-bit Windows (DWORD type + 4 padding + 32-byte union). This struct alignment was a known bug fixed on 2026-03-13 — before the fix, all SendInput calls returned “The parameter is incorrect”.

Mouse coordinates are converted to the Win32 absolute coordinate system (0-65535 range):

absX = x * 65535 / screenW
absY = y * 65535 / screenH

Keyboard events use codeToVK() which maps browser KeyboardEvent.code values (e.g., "KeyA", "Enter", "ArrowUp") to Windows Virtual Key codes. The mapping priority is: code-based (reliable) over key-based (fallback for single characters). Modifier keys (Ctrl, Shift, Alt, Meta/Win) are sent as separate SendInput calls before the main key on keydown, and released after on keyup.

handleInputMessage in session.go processes the JSON, dispatches to the appropriate InputInjector method, and logs errors at debug level (so individual injection failures do not crash the session).

Your Test Setup

Prerequisites:

• Remote desktop session active (status "active") between Win-A and Win-B
• Win-B’s desktop is visible and unlocked
• Notepad or a text editor open on Win-B’s desktop for keyboard testing
• No screen saver or UAC prompt active on Win-B (UAC elevates to a different desktop where SendInput does not reach)

ST1 — Mouse Movement and Position Accuracy

What it verifies: Mouse mousemove events from the canvas are scaled correctly to Win-B’s resolution and the cursor on Win-B moves to the corresponding position.

Steps:

1. On ⊞ Win-A , establish an active remote desktop session to Win-B.

2. Click the canvas to give it keyboard focus.

3. Move the mouse to the top-left corner of the canvas. Observe the cursor on Win-B (visible in the canvas) moving to the top-left of Win-B’s desktop.

4. Move the mouse to the bottom-right corner of the canvas. The cursor on Win-B should move to the bottom-right of its display.

5. Test coordinate accuracy at the canvas center. The canvas center (50%, 50%) should map to Win-B’s display center. Open the Windows “Magnifier” app on Win-B and zoom in to verify cursor precision.

6. Open browser DevTools console on ⊞ Win-A and verify mousemove events are being sent. Add a temporary log:

// In console — intercept canvas events (read-only inspection)
document.querySelector('canvas').addEventListener('mousemove', (e) => {
    const canvas = e.target;
    const rect = canvas.getBoundingClientRect();
    const rawX = e.clientX - rect.left;
    const rawY = e.clientY - rect.top;
    // hostResolution is in component state; approximate from canvas natural dimensions:
    const scaleX = canvas.width / rect.width;
    const scaleY = canvas.height / rect.height;
    console.log('Canvas raw:', rawX.toFixed(0), rawY.toFixed(0),
                '=> Host:', Math.round(rawX * scaleX), Math.round(rawY * scaleY));
}, { passive: true });

Move the mouse to a known position (e.g., top-left corner of Win-B’s taskbar) and verify the “Host” coordinates match the expected pixel position.

7. On ⊞ Win-B , enable cursor position display (optional). Open PowerShell:

Add-Type -AssemblyName System.Windows.Forms
while ($true) {
    $pos = [System.Windows.Forms.Cursor]::Position
    Write-Host "Cursor: $($pos.X), $($pos.Y)" -NoNewline
    Write-Host "`r" -NoNewline
    Start-Sleep -Milliseconds 100
}

Move the mouse in the viewer and verify Win-B’s reported cursor position matches.

Pass: Win-B cursor moves in sync with the viewer mouse; corner positions map correctly; coordinate scaling is proportional to canvas size.

Fail / Common issues:

• Cursor moves in the wrong direction or to wrong position — scaleCoordinates() relies on hostResolution being set from the "resolution" control message. If the control message was not received, hostResolution defaults to 1280x720, causing incorrect scaling on a 1920x1080 display.
• No cursor movement on Win-B — the input DataChannel may not be open. Check that connState === "connected" in the viewer. The sendInput function returns early if inputChannelRef.current?.readyState !== "open".
• Cursor moves but position drifts — the canvas is zoomed/scaled by CSS (max-w-full max-h-full object-contain). getBoundingClientRect() returns the CSS-rendered size, not the canvas pixel size. scaleCoordinates accounts for this; if position is off, verify canvas.getBoundingClientRect() returns the visual size.

ST2 — Mouse Button Clicks (Left, Right, Middle)

What it verifies: Mouse button down/up events are injected via SendInput with the correct Win32 mouse flags.

Steps:

1. On ⊞ Win-A , click the canvas to focus it.

2. Left click test: Single left-click on Win-B’s desktop (visible in canvas). A file or icon should become selected on Win-B.

3. Double-click test: Double-click an item on Win-B’s desktop. It should open (e.g., double-clicking the Recycle Bin should open it). Note: double-click is two mousedown/mouseup pairs in rapid succession — no special handling is needed in the viewer.

4. Right-click test: Right-click anywhere on Win-B’s desktop. Win-B’s context menu should appear (visible in the canvas). Note that onContextMenu on the canvas calls e.preventDefault() to suppress the viewer’s own context menu — only Win-B’s context menu should appear.

5. Middle-click test (if supported): Middle-click on an open browser tab on Win-B to close it. On Win32, mousefMiddleDown / mousefMiddleUp flags are used in MouseDown/MouseUp.

6. Dismiss any open menus on Win-B by left-clicking an empty area of the desktop.

7. Verify via agent log on Win-B that SendInput calls are succeeding (no debug errors):

Get-Content "$env:LOCALAPPDATA\Programs\QuickZTNA\quickztna-svc.log" -Tail 50 |
    Select-String "input injection error|SendInput"

If the struct alignment bug were present, you would see "SendInput mouse: The parameter is incorrect" for every click. The fix (40-byte inputUnion struct) means this should be absent.

Pass: Left-click selects, double-click opens, right-click shows Win-B’s context menu; no “SendInput error” lines in agent log.

Fail / Common issues:

• Right-click shows the viewer’s context menu instead of Win-B’s — the onContextMenu handler on the canvas is not calling e.preventDefault(). This would be a regression.
• Middle-click not working — some mice do not fire onMouseDown with button: 1. Verify the browser is correctly reporting the middle button.
• "input injection error" in agent log — if SendInput is returning an error, verify the inputUnion struct size. The correct size is 40 bytes. Check that the service binary was built from the fixed input_windows.go.

ST3 — Keyboard Keys: Alphanumeric and Special Keys

What it verifies: Letter keys, digit keys, Enter, Tab, Escape, Backspace, Delete, arrow keys, and function keys are mapped correctly by codeToVK() and injected via Win32 SendInput with inputKeyboard.

Steps:

1. On ⊞ Win-B , open Notepad (Win + R → notepad). This gives a clear text field for keyboard testing.

2. On ⊞ Win-A , click the canvas to focus it.

3. Alphanumeric test: Type a phrase: Hello World 123. Verify on Win-B that the text appears in Notepad. Note that the codeToVK() mapping handles:

   • Letter keys via code[3] char from "KeyA"-"KeyZ" mapping to VK codes 0x41-0x5A
   • Digit keys via code[5] char from "Digit0"-"Digit9" mapping to VK codes 0x30-0x39

4. Enter key: Press Enter in the canvas. A new line should appear in Notepad on Win-B. codeToVK("Enter", "Enter") returns 0x0D (VK_RETURN).

5. Backspace: Press Backspace. The last character typed should be deleted. codeToVK("Backspace", ...) returns 0x08 (VK_BACK).

6. Arrow keys: Press the left, right, up, down arrow keys. The Notepad cursor should move. Codes "ArrowLeft", "ArrowRight", "ArrowUp", "ArrowDown" map to 0x25, 0x27, 0x26, 0x28.

7. Function keys: Press F5 in Notepad (this inserts the current time/date in Notepad). codeToVK("F5", "F5") returns 0x74 (VK_F5 = 0x70 + 5 - 1).

8. Escape: Press Escape. If a menu is open on Win-B, it should close. Code "Escape" maps to 0x1B.

9. Tab: In Notepad, Tab should insert a tab character. Code "Tab" maps to 0x09.

Pass: All typed characters appear in Notepad exactly as typed; Enter creates a new line; Backspace deletes; arrow keys move the cursor; F5 inserts timestamp; Escape dismisses menus.

Fail / Common issues:

• Uppercase letters appear as lowercase (or vice versa) — the VK code for letters is the uppercase ASCII code (0x41-0x5A). Win32 uses Shift state to determine case. Without Shift, VK_A produces lowercase ‘a’ in most apps. Type with Shift to get uppercase.
• Digits not working — "Digit1" through "Digit9" are mapped; "Digit0" is also mapped. If digits from the numpad are used, the code is "Numpad0" etc., which is not in the mapping — only the main keyboard row digits are supported.
• codeToVK returns 0 for a key — the key code is not in the mapping. Check input_windows.go:codeToVK. Special characters (punctuation, brackets, etc.) are not explicitly mapped and fall through to the single-char key fallback only for A-Z and 0-9.

ST4 — Modifier Key Combinations

What it verifies: Ctrl, Shift, Alt, and Meta (Win) key combinations are correctly sent with the main key and produce the expected system behavior on Win-B.

Steps:

1. On ⊞ Win-A , with focus on the canvas and Notepad open on Win-B:

2. Ctrl+A (Select All): Press Ctrl+A. All text in Notepad on Win-B should be selected.

   The viewer sends:

   • keydown: { key: "Control", code: "ControlLeft", modifiers: [] } — not in codeToVK, returns 0, ignored
   • keydown: { key: "a", code: "KeyA", modifiers: ["ctrl"] } — codeToVK returns 0x41 (VK_A)

   The agent KeyDown("a", "KeyA", ["ctrl"]) calls modifierToVK("ctrl") → 0x11, sends SendInput(VK_CONTROL) first, then SendInput(VK_A).

3. Ctrl+C (Copy): With text selected, press Ctrl+C. Win-B’s clipboard should now contain the selected text.

4. Ctrl+V (Paste): Click at the end of the text in Notepad and press Ctrl+V. The copied text should be pasted.

5. Ctrl+Z (Undo): Press Ctrl+Z. The paste should be undone.

6. Shift+Letter (Uppercase): Type Shift+H to produce uppercase ‘H’. The viewer sends modifiers ["shift"], agent sends SendInput(VK_SHIFT) then SendInput(VK_H).

7. Alt+F4 (Close Window): Press Alt+F4. Notepad’s window should close on Win-B.

   Warning: Alt+F4 will close the front-most application on Win-B. Make sure you close Notepad intentionally (save or discard when prompted).

8. Win key (Meta): Press the Meta/Windows key on Win-A’s keyboard (the key labeled with the Windows logo). modifierToVK("meta") → 0x5B (VK_LWIN). Win-B’s Start menu should open.

   Press Escape to close it (or Meta again).

Pass: Ctrl+A selects all; Ctrl+C/V copies and pastes; Shift produces uppercase; Alt+F4 closes the window; Win key opens Start menu.

Fail / Common issues:

• Ctrl+A does not select all — browser may be consuming Ctrl+A as a browser shortcut before the keydown handler fires. Ensure the canvas has focus (tabIndex={0} and user clicked it). The onKeyDown handler calls e.preventDefault() to block browser defaults.
• Meta (Win) key not working — some browsers block KeyboardEvent for the Win/Meta key at the OS level. The key event may not reach the viewer’s onKeyDown handler. This is a browser limitation, not a code bug.
• Modifier released too early — if the user releases Ctrl before ‘a’, the viewer sends keyup for Ctrl before keydown for ‘a’. The agent processes these in order (ordered DataChannel), so the modifier is released before the key press reaches Win32. Type Ctrl+A in a single chord (hold Ctrl, press A, release both).

ST5 — Scroll Wheel

What it verifies: Scroll wheel events (WheelEvent) are transmitted as type: "scroll" with deltaX/deltaY values and injected via Win32 mousefWheel flag.

Steps:

1. On ⊞ Win-B , open a document or webpage with scrollable content (e.g., a long text file in Notepad, or a webpage in a browser).

2. On ⊞ Win-A , hover the mouse over the canvas (no click needed).

3. Scroll up and down using the mouse wheel. Win-B’s content should scroll accordingly.

4. The viewer’s handleWheel callback:

   • Calls e.preventDefault() to stop the viewer’s page from scrolling
   • Sends { type: "scroll", deltaX: Math.round(e.deltaX), deltaY: Math.round(e.deltaY) }

5. The agent’s Scroll(deltaX, deltaY) implementation in input_windows.go:

   • Only sends a mousefWheel event if deltaY != 0 (horizontal scroll deltaX is not forwarded)
   • The Win32 wheel delta is mouseData: uint32(int32(deltaY)) — positive is scroll down, negative is scroll up

6. Verify horizontal scroll is not implemented — scroll left/right on Win-A (if using a trackpad or horizontal scroll wheel). No horizontal scrolling should occur on Win-B (deltaX is ignored).

7. Test fast scroll — spin the wheel rapidly. Win-B should scroll smoothly, not stutter. Since the input DataChannel is ordered: true, scrolls arrive in sequence.

8. Verify via agent log that scroll events are being processed (no errors):

Get-Content "$env:LOCALAPPDATA\Programs\QuickZTNA\quickztna-svc.log" -Tail 30 |
    Select-String "input injection error"

Pass: Win-B content scrolls vertically in sync with the viewer’s mouse wheel; horizontal scroll is a no-op; no injection errors in agent log.

Fail / Common issues:

• Win-B scrolls but the viewer page also scrolls — e.preventDefault() in handleWheel is not being called, or the event listener is passive: true. Check that onWheel on the canvas is not marked passive.
• Scroll direction reversed — WheelEvent.deltaY is positive when scrolling down in browsers, and Win32 MOUSEEVENTF_WHEEL positive value scrolls up. The agent passes uint32(int32(deltaY)) which does a sign-preserving cast. If the scroll direction is wrong on Win-B, the deltaY sign convention may differ between the browser and Win32.
• No scroll on Win-B — the focused application on Win-B may not be under the mouse cursor. Win32 SendInput with mousefWheel sends the scroll to the foreground window, not necessarily the one under the cursor. Move the cursor over the scrollable content on Win-B first.

Summary