TRAMP-RPC

A high-performance TRAMP backend for Emacs that uses a binary RPC server instead of parsing shell command output.

Overview

Traditional TRAMP works by piping shell commands over SSH and parsing their output. This approach is robust but slow, especially for operations that require many round-trips (like directory listings or VC operations).

TRAMP-RPC replaces this with a lightweight Rust server that runs on the remote host. Emacs communicates with it using MessagePack-RPC over SSH, resulting in significantly faster file operations.

Why TRAMP-RPC?

Aspect	Original TRAMP	TRAMP-RPC
Communication	Shell commands + parsing	MessagePack-RPC protocol
Latency	Multiple round-trips	Single round-trip
Batching	Not supported	Multiple ops per request
Shell dependency	Required on remote	Not needed
Binary required	None	~850KB Rust server

Features

Fast file operations via binary RPC protocol (2-38x faster than shell-based TRAMP)
Async process support (make-process, start-file-process)
Full VC mode integration (git, etc.)
Magit/Projectile optimizations with parallel git command prefetch
Automatic binary deployment (download or build from source)
Support for Linux and macOS (x86_64 and aarch64)
Batch/pipelined requests for reduced round-trip latency
Multi-hop support via SSH ProxyJump
Filesystem watching with automatic cache invalidation
PTY support for terminal emulators (vterm, eat)

Requirements

Emacs 30.1 or later
Tramp 2.8.1.4 or later (install from GNU ELPA if your Emacs bundles an older version)
msgpack.el 0.1.1 or later (installed automatically from MELPA)
SSH access to remote hosts
Remote host running Linux or macOS (x86_64 or aarch64)

Installation

From NonGNU ELPA (coming soon)

(use-package tramp-rpc
  :ensure t)

From Git (Emacs 30+)

(use-package tramp-rpc
  :after tramp
  :vc (:url "https://github.com/ArthurHeymans/emacs-tramp-rpc"
       :rev :newest
       :lisp-dir "lisp"))

Manual Installation

Clone this repository:

git clone https://github.com/ArthurHeymans/emacs-tramp-rpc.git

Add to your Emacs init file:

(add-to-list 'load-path "/path/to/emacs-tramp-rpc/lisp")
(require 'tramp-rpc)

For Doom Emacs

In packages.el:

(package! msgpack)
(package! tramp-rpc :recipe (:host github :repo "ArthurHeymans/emacs-tramp-rpc" :files ("lisp/*.el")))

In config.el:

(use-package! msgpack)
(use-package! tramp-rpc)

From Nix flake

Include this repository as a flake input and add its overlay to nixpkgs:

inputs = {
  nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
  emacs-tramp-rpc.url = "github:ArthurHeymans/emacs-tramp-rpc";
};

outputs = inputs@{ nixpkgs, emacs-tramp-rpc, ... }:
  let
    system = "x86_64-linux";
    inherit (nixpkgs) lib;
    pkgs = import nixpkgs {
      inherit system;
      overlays = [ emacs-tramp-rpc.overlays.default ];
    };
  in ...;

Include tramp-rpc as you would any Elisp package from the emacsPackages scope, e.g. in Home Manager’s programs.emacs.extraPackages:
```
programs.emacs.extraPackages = epkgs: [ epkgs.tramp-rpc ];
    
```

Server binaries for x86_64-linux and aarch64-linux are automatically built and included in the Emacs package derivation; if you would like to change which systems are built, override the archs input:

programs.emacs.extraPackages = epkgs: [
  (epkgs.tramp-rpc.override {
    archs = [
      pkgs.pkgsCross.riscv64-musl
      (import nixpkgs {
        inherit system;
        crossSystem = lib.systems.elaborate {
          config = "armv6l-unknown-linux-musleabihf";
        };
      })
    ];
  })
];

Usage

Access remote files using the rpc method:

/rpc:user@host:/path/to/file

On first connection, the server binary is automatically obtained and deployed:

Release/package installs: download from GitHub Releases first (fastest, ~850KB download), then build from source if Rust is installed and download fails.
Git checkout installs: reuse a fresh source build or source-tree keyed cache. Automatic file operations do not prompt when a binary is missing; run M-x tramp-rpc-deploy-install-binary to choose whether to download a checksum-verified release binary, build the checkout with Cargo, or skip. A downloaded fallback remains keyed to that source tree, while strict build policy uses a separate identity.

The binary is cached locally in ~/.emacs.d/tramp-rpc/ and deployed to ~/.cache/emacs/tramp-rpc/ on the remote host.

Deployment Commands

Command	Description
`M-x tramp-rpc-deploy-install-binary`	Choose and deploy a missing git-checkout binary; use `C-u` to replace one
`M-x tramp-rpc-deploy-status`	Show binary deployment status
`M-x tramp-rpc-deploy-clear-cache`	Clear local binary cache
`M-x tramp-rpc-deploy-remove-binary`	Remove binary from remote

Architecture

┌─────────────┐   SSH/MessagePack-RPC  ┌──────────────────┐
│   Emacs     │ ◄────────────────────► │ tramp-rpc-server │
│ (tramp-rpc) │                        │     (Rust)       │
└─────────────┘                        └──────────────────┘

Module Organization

The Emacs Lisp client is organized into focused modules:

Module	Lines	Purpose
`tramp-rpc.el`	~2450	Core RPC communication & file handlers
`tramp-rpc-process.el`	~1050	Async process & PTY support
`tramp-rpc-magit.el`	~780	Magit/Projectile optimizations & caching
`tramp-rpc-deploy.el`	~950	Binary deployment & version management
`tramp-rpc-advice.el`	~330	Advice functions for process/VC integration
`tramp-rpc-protocol.el`	~190	MessagePack-RPC protocol implementation

Core Module (`tramp-rpc.el`)

The core module provides:

SSH ControlMaster connection management
RPC call/batch/pipeline primitives
Direnv environment caching for processes
All TRAMP file handler operations (file-exists-p, file-attributes, insert-file-contents, etc.)
File name handler registration and dispatch

Process Module (`tramp-rpc-process.el`)

Handles all process-related functionality:

Async pipe processes (make-process, start-file-process)
RPC-based PTY processes for terminal emulators
Terminal resize handling for vterm/eat/shell-mode
Process I/O queuing with async callback-based reading
Adaptive polling for long-running processes

Magit Module (`tramp-rpc-magit.el`)

Optimizations for git/magit/projectile on remote hosts:

Parallel git command prefetch via commands.run_parallel RPC (sends ~60+ git commands in a single round-trip)
TTL-based caches for file-exists and file-truename with max-size eviction
Filesystem watch management via server push notifications for cache invalidation
Process-file cache for serving git commands from prefetched data
Ancestor directory scanning for fast project/VC root detection
Projectile integration (force git ls-files, alien indexing for remote)

Advice Module (`tramp-rpc-advice.el`)

Centralizes all advice functions:

Process I/O: process-send-string, process-send-region, process-send-eof
Process info: signal-process, process-status, process-exit-status
Process metadata: process-command, process-tty-name
VC integration: vc-call-backend for proper default-directory handling
Eglot integration: Bypass shell wrapping for RPC connections

RPC Server Methods

The dispatcher exposes these public methods:

Category	Method	Description
Batch	`batch`	Execute up to 64 requests with bounded concurrency.
File	`file.stat`	Return file metadata.
File	`file.truename`	Resolve a file’s canonical path.
Directory	`dir.list`	List directory entries, optionally with attributes.
Directory	`dir.create`	Create a directory.
Directory	`dir.remove`	Remove a directory.
File I/O	`file.read`	Read file bytes.
File I/O	`file.write`	Write file bytes.
File I/O	`file.copy`	Copy a file.
File I/O	`file.rename`	Rename a file.
File I/O	`file.delete`	Delete a file.
File I/O	`file.set_modes`	Set file mode bits.
File I/O	`file.set_times`	Set file timestamps.
File I/O	`file.make_symlink`	Create a symbolic link.
File I/O	`file.make_hardlink`	Create a hard link.
File I/O	`file.chown`	Change file ownership.
Process	`process.run`	Run a command synchronously.
Process	`process.start`	Start a managed pipe process.
Process	`process.write`	Write to managed process stdin.
Process	`process.read`	Read managed process output.
Process	`process.status`	Return managed process status.
Process	`process.close_stdin`	Close managed process stdin.
Process	`process.kill`	Signal a managed process.
Process	`process.list`	List managed processes.
PTY	`process.start_pty`	Start a pseudo-terminal process.
PTY	`process.read_pty`	Read pseudo-terminal output.
PTY	`process.write_pty`	Write pseudo-terminal input.
PTY	`process.resize_pty`	Resize a pseudo-terminal.
PTY	`process.kill_pty`	Signal a pseudo-terminal process.
PTY	`process.close_pty`	Close a pseudo-terminal process.
PTY	`process.list_pty`	List pseudo-terminal processes.
System	`system.info`	Return server and host information.
System	`system.getenv`	Read an environment variable.
System	`system.expand_path`	Expand home-directory paths.
System	`system.statvfs`	Return filesystem capacity information.
System	`system.groups`	Return supplementary groups.
Commands	`commands.run_parallel`	Run commands concurrently.
Commands	`ancestors.scan`	Scan ancestor directories.
High-level	`highlevel.test_files_in_dir`	Find named files in a directory.
High-level	`highlevel.locate_dominating_file_multi`	Find an ancestor containing one of several names.
High-level	`highlevel.dir_locals_find_file_cache_update`	Update directory-local file cache data.
Watch	`watch.add`	Add a filesystem watch.
Watch	`watch.remove`	Remove a filesystem watch.
Watch	`watch.list`	List filesystem watches.

Binary Deployment

How It Works

User connects via /rpc:host:/path
         │
         ▼
Remote already has binary? ──yes──► Done
         │ no
         ▼
Check local cache (~/.emacs.d/tramp-rpc/VERSION/ARCH/)
         │
         ├─ Found ──────────────────► Transfer to remote
         │
         ▼
Download from GitHub Releases
         │
         ├─ Success ────────────────► Cache locally, transfer to remote
         │
         ▼
Build with cargo (if Rust installed)
         │
         ├─ Success ────────────────► Cache locally, transfer to remote
         │
         ▼
Error with helpful instructions

Supported Platforms

Platform	Architecture	Status
Linux	x86_64	✓
Linux	aarch64	✓
Linux	i686	✓
Linux	armv7	✓
Linux	armv5te	✓
Linux	arm/ARMv6	✓
macOS	x86_64	✓
macOS (Apple Silicon)	aarch64	✓

Manual Binary Installation

If automatic deployment fails, download manually from GitHub Releases, extract the archive, and place the binary on the remote host at:

~/.cache/emacs/tramp-rpc/tramp-rpc-server-VERSION

For example:

~/.cache/emacs/tramp-rpc/tramp-rpc-server-0.9.0

Building from Source

Using Nix (recommended)

# Build for current platform
nix build

# Cross-compile for specific target
nix build .#tramp-rpc-server-x86_64-linux
nix build .#tramp-rpc-server-aarch64-linux

# Development shell with all tools
nix develop

Using Cargo

cd server
cargo build --release

The binary will be at target/release/tramp-rpc-server.

Cross-compilation with Cargo

# Install target
rustup target add aarch64-unknown-linux-gnu

# Build (requires appropriate linker)
cargo build --release --target aarch64-unknown-linux-gnu

Configuration

;; Prefer building from source over downloading for release/package installs
;; (default: nil)
(setq tramp-rpc-deploy-prefer-build t)

;; Git checkout policy (default: auto):
;; - auto: reuse source-keyed artifacts; use
;;   M-x tramp-rpc-deploy-install-binary when one must be obtained;
;;   use C-u M-x tramp-rpc-deploy-install-binary to replace an existing one
;; - build: strictly build from source, using a build-only binary identity
;; - release: use release-version binaries and paths
(setq tramp-rpc-deploy-git-build-policy 'build)

;; Local cache directory (default: ~/.emacs.d/tramp-rpc/)
(setq tramp-rpc-deploy-local-cache-directory "~/.cache/emacs/tramp-rpc-binaries")

;; Remote installation directory (default: ~/.cache/emacs/tramp-rpc)
(setq tramp-rpc-deploy-remote-directory "~/.local/bin/tramp-rpc")

;; Disable automatic deployment (default: t)
(setq tramp-rpc-deploy-auto-deploy nil)

;; Use different GitHub repo for downloads
(setq tramp-rpc-deploy-github-repo "myuser/my-fork")

;; Download timeout in seconds (default: 120)
(setq tramp-rpc-deploy-download-timeout 60)

Deployment fallback policy

When the expected remote binary already exists and is executable, TRAMP-RPC reuses it if no trusted local artifact can be obtained (for example because download, build, or local-cache access is unavailable). This fallback never covers a missing remote binary. Whenever a trusted local artifact is available, TRAMP-RPC compares SHA256 digests and either reuses the match or, with automatic deployment enabled, replaces a mismatch through the verified staging-and-activation operation. With automatic deployment disabled, a verified mismatch remains an explicit error.

Process compatibility notes

RPC PTYs disable kernel ECHO, ECHONL, and ONLCR. Emacs terminal consumers such as comint and eat perform local echo and line handling; code that relies directly on kernel echo or CRLF conversion will observe different PTY semantics.
RPC PTY writes wait for the remote write acknowledgement and can therefore block for up to the RPC timeout when the remote program stops reading. Pipe writes remain queued by default; set tramp-rpc-synchronous-pipe-writes non-nil to make them synchronous too.

Troubleshooting

Check deployment status

Run M-x tramp-rpc-deploy-status to see:

Current version
Local architecture
Whether Rust/cargo is available
Cached binaries
Download URLs
The binary ID (in case you are manually installing the binary on the remote out of band)

diff-hl issues in dired

If you experience issues with diff-hl in dired buffers on remote hosts:

(setq diff-hl-disable-on-remote t)

Connection issues

The server binary is deployed using standard SSH (scpx method by default). Ensure you can connect to the remote host with:

ssh -o BatchMode=yes user@host echo success

Download failures

If GitHub downloads fail (corporate firewall, etc.), you can:

Install Rust and let tramp-rpc build locally:

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

Download manually and place in the cache directory (see above)
Pre-deploy to remote hosts using your own method

Protocol

TRAMP-RPC uses MessagePack-RPC over stdin/stdout with length-prefixed binary framing.

Protocol History

TRAMP-RPC originally used JSON-RPC with newline-delimited messages. This was changed to MessagePack-RPC for several reasons:

Aspect	JSON-RPC (old)	MessagePack-RPC (current)
Binary data	Base64 encoded (~33% overhead)	Native binary type (no overhead)
Message framing	Newline-delimited	Length-prefixed binary
Non-UTF8 filenames	Required escaping/encoding	Native binary support
Boolean false	JSON `false`	MessagePack false (0xc2)
Message size	Larger (text format)	~33% smaller (binary format)

The switch eliminates encoding overhead for file transfers and fixes edge cases with non-UTF8 filenames that are valid on Unix filesystems.

Why MessagePack?

MessagePack is a binary serialization format that provides:

Native binary data support (no base64 encoding needed for file content)
~33% smaller messages compared to JSON
Faster serialization/deserialization
Proper distinction between null and false values

Framing

Each message is prefixed with a 4-byte big-endian length:

<4-byte length><MessagePack payload>

Message Format

Request (conceptual structure):

((version . "2.0")
 (id . 1)
 (method . "file.stat")
 (params . ((path . "/etc/passwd"))))

Response (conceptual structure):

((version . "2.0")
 (id . 1)
 (result . ((type . "file")
            (size . 2847)
            (mode . 420))))

Binary Data

File content, paths, and process I/O are transmitted as raw binary (MessagePack bin type), eliminating encoding overhead and ensuring correct handling of:

Non-UTF8 filenames
Binary file content
Arbitrary byte sequences in process output

Performance

TRAMP-RPC significantly outperforms traditional TRAMP for most operations:

Operation	RPC (median)	SSH (median)	Speedup
connection-setup	31 ms	1.17 s	38.2x
file-exists	3.3 ms	38.8 ms	11.9x
file-attributes	3.4 ms	23.3 ms	6.8x
dir-files-and-attrs	3.5 ms	95.9 ms	27.1x
file-read	7.7 ms	20.2 ms	2.6x
file-write	74.2 ms	219.9 ms	3.0x
directory-files	12.1 ms	37.2 ms	3.1x
copy-file	43.1 ms	192.0 ms	4.5x
10x file-attributes	37.6 ms	304.7 ms	8.1x

Batch operations provide additional 2-4x speedup by combining multiple requests into a single round-trip (e.g., 10x file.stat drops from 37.6 ms sequential to 9.1 ms batched).

For detailed benchmarks and an in-depth technical comparison with original TRAMP, see Technical Comparison.

Testing

TRAMP-RPC includes a comprehensive test suite using Emacs ERT (Emacs Lisp Regression Testing).

Test Categories

Category	Tests	Requirements
Protocol	8	None (pure Elisp)
Conversion	2	None (pure Elisp)
Server Integration	4	RPC server binary
Multi-hop	21	None (pure Elisp)
Autoload	8	None (pure Elisp)
Remote File Ops	53	SSH + RPC server

Running Tests

Quick Protocol Tests (no dependencies)

./test/run-tests.sh --protocol

Or directly with Emacs:

emacs -Q --batch -l test/tramp-rpc-mock-tests.el \
  --eval "(ert-run-tests-batch-and-exit \"^tramp-rpc-mock-test-protocol\")"

All Mock Tests (includes server integration)

./test/run-tests.sh --mock

This runs protocol tests plus server integration tests that communicate directly with the RPC server (no SSH needed).

Full Remote Tests (requires SSH)

TRAMP_RPC_TEST_HOST=your-remote-host ./test/run-tests.sh --remote

Or:

emacs -Q --batch \
  -l test/tramp-rpc-tests.el \
  --eval "(setq tramp-rpc-test-host \"your-remote-host\")" \
  --eval "(ert-run-tests-batch-and-exit \"^tramp-rpc-test\")"

Test Files

test/tramp-rpc-tests.el - Full ERT test suite for remote operations
test/tramp-rpc-mock-tests.el - CI-compatible tests (no SSH required)
test/tramp-rpc-autoload-tests.el - Autoload mechanism tests
test/run-tramp-tests.el - Upstream tramp-tests.el against tramp-rpc backend
test/run-tests.sh - Test runner script
test/run-autoload-tests.sh - Autoload test runner

CI Integration

The GitHub Actions workflow runs:

Rust Build - Builds for 4 targets (x86_64/aarch64 Linux/macOS) with format check
Elisp Byte-compile - Verifies all .el files compile without errors
Autoload Tests - Verifies method registration and handler setup
Protocol Tests - MessagePack-RPC encoding/decoding (no server)
Multi-hop Tests - ProxyJump conversion, connection keys, hop normalization
Server Integration Tests - Direct server communication (Rust binary)
Full Test Suite - Complete tests via SSH to localhost
Upstream TRAMP Tests - Runs tramp-tests.el against tramp-rpc backend

License

This project is licensed under the GNU General Public License v3.0 or later - see the LICENSE file for details.

Contributing

Contributions welcome! Please ensure code passes cargo clippy and cargo test before submitting.

For Emacs Lisp changes, also run the test suite:

./test/run-tests.sh --mock

Name		Name	Last commit message	Last commit date
.cargo		.cargo
.github/workflows		.github/workflows
benchmark		benchmark
doc		doc
lisp		lisp
scripts		scripts
server		server
test		test
.gitignore		.gitignore
AGENTS.md		AGENTS.md
CLAUDE.md		CLAUDE.md
Cargo.lock		Cargo.lock
Cargo.toml		Cargo.toml
LICENSE		LICENSE
README.org		README.org
default.nix		default.nix
flake.lock		flake.lock
flake.nix		flake.nix

Latest commit

History

Folders and files

Repository files navigation