Yes, you should set ServerName:
http://wiki.apache.org/httpd/CouldNotDetermineServerName
http://httpd.apache.org/docs/current/mod/core.html#servername
You can find information on the layouts used by the various httpd distributions here:
http://wiki.apache.org/httpd/DistrosDefaultLayout
In your case the file to edit is /etc/httpd/conf/httpd.conf