You're not missing anything. RestTemplate#exchange(..)
is the appropriate method to use to set request headers.
Here's an example (with POST, but just change that to GET and use the entity you want).
Note that with a GET, your request entity doesn't have to contain anything (unless your API expects it, but that would go against the HTTP spec). It can be an empty String.