In addition to the other responses:
Once obtained, Access Tokens are typically sent along with every request from Clients to protected Resource Servers. This induce a risk for access token stealing and replay (assuming of course that access tokens are of type "Bearer" (as defined in the initial RFC6750).
Examples of those risks, in real life:
Resource Servers generally are distributed application servers and typically have lower security levels compared to Authorization Servers (lower SSL/TLS config, less hardening, etc.). Authorization Servers on the other hand are usually considered as critical Security infrastructure and are subject to more severe hardening.
Access Tokens may show up in HTTP traces, logs, etc. that are collected legitimately for diagnostic purposes on the Resource Servers or clients. Those traces can be exchanged over public or semi-public places (bug tracers, service-desk, etc.).
Backend RS applications can be outsourced to more or less trustworthy third-parties.
The Refresh Token, on the other hand, is typically transmitted only twice over the wires, and always between the client and the Authorization Server: once when obtained by client, and once when used by client during refresh (effectively "expiring" the previous refresh token). This is a drastically limited opportunity for interception and replay.
Last thought, Refresh Tokens offer very little protection, if any, against compromised clients.