What you will set up
Hermes Agent is an open-source AI agent from Nous Research. It connects to an AI model at the provider you choose, runs commands, searches the web, remembers what it learns about you, and runs scheduled tasks. You talk to it from a chat app. On a server it keeps running when your laptop is closed.
Here it runs in rootless Podman under a user of its own called hermes, and you talk to it over Matrix, end-to-end encrypted, from the Matrix guide. The commands it runs stay inside its container, under that unprivileged user, so the agent cannot touch your other apps. It opens no ports: it only connects out, to the model provider and to Matrix.
Every step below was run on a Melonslab VC-P Alloy (2 vCPU, 8 GB) with Debian 13, with the Matrix server from our guide on the same machine:
- In an encrypted chat in Element, the agent answered, ran
uname -rwhen asked, and remembered a fact in a fresh session. - From inside its container, an app that listens only on the server's loopback address could not be reached.
- With OpenRouter's free models, the API key's usage stayed at zero.
- Everything came back by itself after a reboot.
Hermes used about 290 MB of memory. Its image takes 2.8 GB of disk.
Before you start
You need:
- a server set up as in the Podman guide, ideally with ufw from the security guide. Hermes needs no open ports;
- a Matrix server as in the Matrix guide, and your own account on it;
- an API key from a model provider. This guide uses OpenRouter, which has free models: create a key at openrouter.ai/keys, and give it a spending limit.
The examples use example.com for your Matrix domain, matrix.example.com for the Matrix server and @anna:example.com for your own account. Replace them throughout.
1. Create the user
As root:
useradd -m -s /bin/bash hermes
loginctl enable-linger hermes
2. Create a Matrix account for the agent
The agent gets a Matrix account of its own. As root, switch to machinectl shell matrix@, and create it, answering no to the admin question:
podman exec -it matrix-synapse register_new_matrix_user -c /data/homeserver.yaml http://localhost:8008
Name it hermes. Hermes logs in with an access token rather than the password, so that it keeps one device for its encryption keys. Get one, with the password you just set in place of BOT_PASSWORD:
curl -s -X POST https://matrix.example.com/_matrix/client/v3/login \
-H 'Content-Type: application/json' \
-d '{"type": "m.login.password", "identifier": {"type": "m.id.user", "user": "hermes"}, "password": "BOT_PASSWORD", "initial_device_display_name": "Hermes Agent"}'
The answer contains "access_token":"...". Copy the value, and keep it private: it is the agent's login.
3. Describe the container
Go back to root with exit, then switch to machinectl shell hermes@:
mkdir -p ~/.config/containers/systemd
Create ~/.config/containers/systemd/hermes.container:
[Unit]
Description=Hermes Agent
[Container]
ContainerName=hermes
Image=docker.io/nousresearch/hermes-agent:latest
Volume=hermes-data:/opt/data
# The Matrix server runs on this server too: reach it through the host, not the container itself.
AddHost=matrix.example.com:host-gateway
Exec=gateway run
AutoUpdate=registry
[Service]
Restart=always
[Install]
WantedBy=default.target
Everything Hermes keeps lives in the hermes-data volume: its settings, your API key, its memory and its chat sessions. The AddHost line is there because a rootless container shares the server's own address, so without it the agent would try to reach matrix.example.com inside its own container. Leave it out if your Matrix account is on another server, such as matrix.org.
Start it:
systemctl --user daemon-reload
systemctl --user start hermes
systemctl --user enable --now podman-auto-update.timer
The first start downloads the image, about 1 GB, and takes a minute or two.
4. Choose the model
podman exec -it hermes hermes model
Choose OpenRouter with the arrow keys and Enter, and paste your API key when it asks for OPENROUTER_API_KEY. Hermes then lists OpenRouter's models with their prices. Pick one marked free, or choose Enter custom model name and type openrouter/free, which sends each request to one of OpenRouter's free models. Keep the suggested reasoning effort with Enter.
Free models are slower and rate-limited. For more reliable answers, pick a paid model: the spending limit on your key keeps the cost in check. Hermes logs a warning that openrouter/free "may incur real spend", because its name does not end in :free. In our tests its usage stayed at zero.
5. Connect it to Matrix
Give Hermes the server, the token from step 2, and your own address, so that only you can talk to it:
podman exec hermes hermes config set MATRIX_HOMESERVER https://matrix.example.com
podman exec hermes hermes config set MATRIX_ACCESS_TOKEN ACCESS_TOKEN
podman exec hermes hermes config set MATRIX_ALLOWED_USERS @anna:example.com
podman exec hermes hermes config set MATRIX_E2EE_MODE required
podman exec hermes hermes config set MATRIX_RECOVERY_KEY_OUTPUT_FILE /opt/data/matrix-recovery-key.txt
systemctl --user restart hermes
Element encrypts direct chats, so MATRIX_E2EE_MODE required makes the agent handle encryption, and refuse to start without it. On its first start with these settings, Hermes sets up cross-signing for its account, which lets your apps trust its device, and saves the account's recovery key to matrix-recovery-key.txt in its volume.
After half a minute, check that it connected:
podman exec hermes grep "matrix connected" /opt/data/logs/gateway.log
6. Talk to it
In Element, start a direct chat with @hermes:example.com. The agent joins by itself. Its first message is a notice about a home channel, the chat where it sends the results of scheduled tasks: answer !sethome to make this chat that channel.
Then ask it anything. It marks each message with ✅ when it has read it, and replies to it. Ask it to run a command, and it shows the command it ran. Tell it something about yourself, and it saves it to its memory, which stays when you start a fresh session.
Element keeps commands that start with / for itself, so give Hermes its commands with ! instead. !new starts a fresh session, and asks you to confirm with !approve.
7. What the agent can reach
Commands the agent runs stay in its container:
- they run as the
hermesuser, so they cannot read the files of your other apps or of root; - apps that listen only on the server's loopback address, as the apps in our other guides do behind Caddy, cannot be reached from the container;
- the internet, and anything your server serves to the internet, can be reached.
Only the Matrix users in MATRIX_ALLOWED_USERS can give it instructions. Hermes also has a web dashboard, which stores API keys: this guide leaves it off, and Hermes' own documentation warns against exposing it to the internet.
8. Keep it up to date
The timer from step 3 updates the image every day. The latest tag follows Hermes' stable releases. podman auto-update --dry-run shows whether an update is waiting.
9. Back up
As hermes:
mkdir -p ~/backup
systemctl --user stop hermes
podman volume export hermes-data --output ~/backup/hermes-data.tar
systemctl --user start hermes
That saves the settings, the memory and the chat sessions, and also your API key, the Matrix access token and the recovery key. Copy ~/backup to another machine, and keep it private.
Troubleshooting
The agent never answers. Check that MATRIX_ALLOWED_USERS is your full address, starting with @, and read podman exec hermes tail -30 /opt/data/logs/gateway.log.
The log says it cannot reach the homeserver. If your Matrix server runs on this server, the AddHost line from step 3 is missing or names a different host.
Element shows "Unrecognised command" for /new. Use !new.
Answers are slow, or stop for a while. Free models are rate-limited. Pick another model with step 4, or a paid one.