The connection string generated by the project containing the .edmx file generates the connection string, this would appear to be a holdover from the app.config sorts of files that were copied to the output directory and referenced by the executable to store runtime config information.
This breaks in the web project as there is no automatic process to add random .config information into the web.config file for the web project.
Easiest is to copy the connection string from the config file to the connections section of the web.config file and disregard the config file contents.