Comment # 2 on bug 1088564 from Wolfgang Bauer

(In reply to Fabian Vogt from comment #1)
> ssh-agent is never started by itself

It is started by /etc/X11/xdm/sys.xsession if a gpg config is found in the
user's home.

Btw, I think there is another bug report already that this doesn't work on
Wayland, but I'm not sure.