Fix Matrix Synapse “Too Many Open Files” on systemd

Matrix Synapse may start normally and then stop accepting connections, log “Too many open files,” or appear stuck under load. On a systemd installation, the first thing to check is the file-descriptor limit inherited by the running Synapse process. Changing a shell’s ulimit or editing a PAM limits file usually does not change the limit for a service that systemd starts.

The steps below apply to a Linux host where Synapse runs as a systemd service. The unit may be named matrix-synapse.service, matrix-synapse@.service, or something package-specific. Confirm the real unit before copying commands. If Synapse runs in Docker, Kubernetes, or another container, see the separate section below.

What does “Too many open files” mean?

Linux gives each process a limit on open file descriptors. A descriptor can refer to a file, socket, pipe, or other resource; network connections therefore consume descriptors too. When a process reaches its soft limit, an attempt to open or accept another descriptor can fail with EMFILE. Linux also has a separate system-wide file table limit: exhaustion there can produce ENFILE. Raising one Synapse service’s limit will not fix a system-wide shortage.

Synapse’s official administration FAQ says that high connection and federation activity can use many file handles and that exhaustion can make the server stop accepting new TCP connections. Its guidance mentions raising the limit to at least 4,096 in the context of older defaults such as 1,024 or 256. Treat that as a troubleshooting floor for the situation described, not a universal capacity recommendation for every homeserver. Start by checking the effective limit and actual usage.

Linux’s getrlimit and prlimit documentation describes the per-process soft and hard limits. Synapse also has a soft_file_limit setting; the Synapse configuration reference documents a default of zero, which tells Synapse to set its soft limit to the hard limit it can use.

1. Confirm the unit and diagnose the failure

Find the actual service name first. On many package installations it is matrix-synapse.service, but do not assume that name if your installation uses a custom unit or workers.

systemctl list-units --type=service --all | grep -i synapse
systemctl status matrix-synapse.service --no-pager
journalctl -u matrix-synapse.service --since "30 minutes ago" --no-pager

Look for the exact error, the time it began, and whether it appears during a traffic spike, federation activity, or a restart. Next, inspect the limit systemd currently reports:

systemctl show matrix-synapse.service -p LimitNOFILE -p MainPID

LimitNOFILE shows the service manager’s configured limit; it is not a count of descriptors currently open. A shell command such as ulimit -n only reports the current shell’s limit and is not proof of the service’s limit.

2. Raise the systemd service limit with a drop-in

Use a systemd drop-in so package updates do not require editing the vendor unit file. The following value is an example, not a guaranteed ideal for every server. Choose a limit with room above observed peak usage, and check application and dependency compatibility before setting an unusually high value.

sudo systemctl edit matrix-synapse.service

Add this under the editor’s new drop-in:

[Service]
LimitNOFILE=65536

Save and exit, then reload systemd and restart Synapse during an appropriate maintenance window:

sudo systemctl daemon-reload
sudo systemctl restart matrix-synapse.service
systemctl show matrix-synapse.service -p LimitNOFILE

A single LimitNOFILE value sets both the soft and hard limit for the service in systemd. The systemd manual also allows a soft:hard pair. The soft limit cannot exceed the hard limit. Before choosing a very high value, review the systemd.exec documentation, which notes compatibility concerns for programs using the old select(2) interface with descriptors above 1,023. Do not assume every library or plugin in your deployment supports any arbitrary limit.

3. Check Synapse’s own setting and worker services

If your Synapse configuration explicitly sets soft_file_limit to a low number, the application setting may keep the soft limit below the systemd ceiling. Search the active configuration files and included configuration for that key. Either remove an unnecessary override or set it deliberately; the documented default of zero asks Synapse to raise its soft limit to the available hard limit.

For a multi-process deployment, each worker is a separate process and may be started by a separate unit or template instance. Raising the main service’s limit does not automatically prove that every worker has the intended limit. Inspect the actual worker unit names with systemctl list-units, then apply and verify the appropriate drop-in for each service that needs it.

Likewise, settings in /etc/security/limits.conf and a login shell’s ulimit generally do not govern a system service launched directly by systemd. For this setup, change the unit’s resource limit and verify it on the process that systemd actually started.

4. If Synapse runs in a container

A host-level systemd override for a different service does not automatically change a container’s file-descriptor limit. Configure the container runtime or orchestration layer. For Docker Compose, the service definition supports per-container ulimits; an example is:

services:
  synapse:
    ulimits:
      nofile:
        soft: 65536
        hard: 65536

The number is illustrative; size it from usage and the host’s limits. Docker’s Compose service reference documents the ulimits setting. As of Docker Engine 29, Docker’s release notes describe a change in the default nofile limit associated with containerd 2.1.5: containers may receive 1,024 rather than the previous 1,048,576 default. If the issue appeared after an Engine upgrade, check the effective container limit and explicitly set the needed ulimit in the deployment configuration. See the Docker Engine 29 release notes and docker run reference.

5. Verify the limit on the running Synapse process

After restart, check the process-level limit, not only systemd’s configuration. Obtain the main PID and inspect its kernel-reported limits:

PID=$(systemctl show --property=MainPID --value matrix-synapse.service)
grep "Max open files" "/proc/$PID/limits"
find "/proc/$PID/fd" -mindepth 1 -maxdepth 1 | wc -l

If the main PID is zero or belongs to a wrapper rather than Synapse, identify the Synapse process in the service’s cgroup and inspect that PID instead. The descriptor count is a snapshot; repeat it during representative load. Confirm that “Max open files” reflects your intended soft and hard limits, the count stays comfortably below the soft limit, and the journal no longer reports the error.

If the count keeps rising until it reaches the new ceiling, the limit may only be postponing the failure. The Synapse FAQ notes that stuck or slow requests can be relevant when investigating file-handle exhaustion. Review surrounding logs and request processing, compare descriptor counts over time, and investigate a possible leak or unexpectedly persistent connections rather than continuing to raise the cap blindly.

When raising the per-service limit is not enough

If logs report a system-wide file-table exhaustion error such as ENFILE, inspect host-wide usage and other services; a per-service LimitNOFILE change will not increase the kernel’s system-wide capacity. Avoid changing global kernel limits without confirming the error and understanding host-wide impact. If the service’s reported limit remains low after a drop-in, run systemctl cat matrix-synapse.service to confirm the drop-in is loaded, verify the unit name, reload systemd, and restart the correct unit.

Quick self-check

• Did you identify the exact Synapse unit or container that logs the error?
• Does systemctl show report the intended limit for that unit?
• Does /proc/<Synapse-PID>/limits show the intended soft and hard limits after restart?
• Is the descriptor count below the soft limit under normal peak load?
• Are worker units and container settings checked separately where applicable?
• Did the error stop, or is usage continuing to grow toward the new ceiling?

If all process-level checks pass and the error has stopped during representative traffic, the systemd limit change is working. If descriptors continue to accumulate or the system reports a global shortage, keep investigating the source rather than treating a larger limit as the complete fix.

Sources

• Synapse administration FAQ: file-handle exhaustion
• Synapse configuration reference: soft_file_limit
• systemd.exec manual: service resource limits
• Linux manual: getrlimit and prlimit
• Docker Engine 29 release notes