[Architecture] Initial Improvement

[PinJinx]

Addressed Issues:

Fixes #(issue number)

Description

Screenshots/Recordings:

Functional Verification

• Please check off the behaviors verified with this change.

Screen Mirror

• Screen MIrror works.

Authentication

• Connection doesn't work without a valid token.

Basic Gestures

• One-finger tap: Verified as Left Click.

• Two-finger tap: Verified as Right Click.

• Click and drag: Verified selection behavior.

• Pinch to zoom: Verified zoom functionality (if applicable).

Modes & Settings

• Cursor mode: Cursor moves smoothly and accurately.

• Scroll mode: Page scrolls as expected.

• Sensitivity: Verified changes in cursor speed/sensitivity settings.

• Copy and Paste: Verified both Copy and Paste functionality.

• Invert Scrolling: Verified scroll direction toggles correctly.

Advanced Input

• Key combinations: Verified "hold" behavior for modifiers (e.g., Ctrl+C) and held keys are shown in buffer.

• Keyboard input: Verified Space, Backspace, and Enter keys work correctly.

• Glide typing: Verified path drawing and text output.

• Voice input: Verified speech-to-text functionality for full sentences.

• Backspace doesn't send the previous input.

Any other gesture or input behavior introduced:

• New Gestures: Verified any other gesture or input behavior introduced in this PR.

Additional Notes:

Checklist

• My PR addresses a single issue, fixes a single bug or makes a single improvement.

• My code follows the project's code style and conventions

• I have performed a self-review of my own code

• I have commented my code, particularly in hard-to-understand areas

• If applicable, I have made corresponding changes or additions to the documentation

• If applicable, I have made corresponding changes or additions to tests

• My changes generate no new warnings or errors

• I have joined the and I will share a link to this PR with the project maintainers there

• I have read the

• Once I submit my PR, CodeRabbit AI will automatically review it and I will address CodeRabbit's comments.

• Incase of UI change I've added a demo video.

⚠️ AI Notice - Important!
We encourage contributors to use AI tools responsibly when creating Pull Requests. While AI can be a valuable aid, it is essential to ensure that your contributions meet the task requirements, build successfully, include relevant tests, and pass all linters. Submissions that do not meet these standards may be closed without warning to maintain the quality and integrity of the project. Please take the time to understand the changes you are proposing and their impact.

Summary by CodeRabbit

• New Features

  • Added a streamlined WebRTC streaming experience with direct signaling and improved video delivery.
  • Added automatic reconnection with progressive retries when network issues interrupt streaming.
  • Added support for updating input configuration across active connections.
  • Added clearer host controls for starting, stopping, checking status, and managing access tokens.

• Bug Fixes

  • Improved handling of connection failures, ICE negotiation, and frozen video streams.
  • Linux input setup now reports permission problems clearly and fails safely.
  • Improved server logging for easier diagnosis of runtime errors.

[github-actions]

⚠️ This PR has merge conflicts.

Please resolve the merge conflicts before review.

Your PR will only be reviewed by a maintainer after all conflicts have been resolved.

📺 Watch this video to understand why conflicts occur and how to resolve them:
https://www.youtube.com/watch?v=Sqsz1-o7nXk

[coderabbitai]

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: a970f645-798e-4520-9f11-dd412be37f6d

📥 Commits

Reviewing files that changed from the base of the PR and between 42c8509 and 585930b.

⛔ Files ignored due to path filters (1)

• package-lock.json is excluded by !**/package-lock.json

📒 Files selected for processing (16)

• package.json
• src/hooks/useWebRtcStream.ts
• src/server/api/InputPeerConnection.ts
• src/server/api/apiHandlers.ts
• src/server/api/apiState.ts
• src/server/api/getLocalIp.ts
• src/server/drivers/linux/index.ts
• src/server/drivers/linux/structs.ts
• src/server/drivers/mac/structs.ts
• src/server/drivers/windows/structs.ts
• src/server/gstreamer/gstManager.ts
• src/server/gstreamer/hostRunner.ts
• src/server/server.ts
• src/server/webRTC.ts
• src/utils/logger.ts
• vite.config.ts

---

Walkthrough

The PR replaces HTTP/SSE and node-datachannel signaling with WebSocket/WebRTC transport using werift, adds UDP-based GStreamer video forwarding, simplifies server routing and lifecycle management, and adds client reconnection handling.

Changes

WebRTC transport and server lifecycle

Layer / File(s)	Summary
WebRTC transport foundation package.json, src/server/webRTC.ts	Adds werift, WebSocket signaling, UDP RTP forwarding, per-client WebRTC sessions, input data channels, configuration updates, and shutdown handling.
Server routing and media lifecycle src/server/server.ts, src/server/gstreamer/gstManager.ts, vite.config.ts	Consolidates authenticated API dispatch, initializes WebRTC and GStreamer managers, sends VP8 RTP to UDP port 5004, and updates server integration.
Client signaling and retry flow src/hooks/useWebRtcStream.ts	Uses one peer connection and WebSocket for SDP/ICE exchange, with queued ICE, watchdog-based failure detection, exponential retries, and unified cleanup.
Runtime bindings and errors src/server/drivers/*, src/utils/logger.ts	Exports native binding handles, throws a Linux uinput permission error, and serializes console output through Winston.

Estimated code review effort: 4 (Complex) | ~60 minutes

Possibly related PRs

• AOSSIE-Org/Rein#361: Reworks the same HTTP/WebRTC signaling and GStreamer lifecycle components.
• AOSSIE-Org/Rein#363: Also modifies useWebRtcStream.ts reconnection handling.
• AOSSIE-Org/Rein#358: Directly relates to the native input and WebRTC data-channel path.

Suggested labels: Typescript Lang

Poem

I’m a rabbit hopping through the stream,
With WebSockets carrying every gleam.
UDP sends the pixels bright,
Retries bring the link back right.
Native inputs thump and play—
Hoppy packets all the way! 🐇

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

⚔️ Resolve merge conflicts

• Resolve merge conflict in branch Latencyv2

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}