ASP.NET build manager is building the website by going through the folders alphabetically, and for each folder it figures out it dependencies and builds the dependencies first and then the selected folder.
In this case the problematic folder which is ~/Controls, is selected to be built at the beginning, from yet an unknown reason, it builds some of the controls there as a separate assembly instead of inside the same assembly as other controls (seems to be connected to the fact that some controls are dependent on other controls in the same folder).
Then the next folder which is built (~/File-Center/Control) is dependent on the root folder ~/ which is dependent on ~/Controls, so the folder ~/Controls is being built again only this time the controls which were separated to their own assembly are now joined to the same assembly as other controls with the separated assembly still being referenced.
So at this point 2 assembly (at least) have the same controls and the build fails.
Although we still don't know why this happened, we were able to work around it by changing the Controls folder name to ZControls, this way it is not built before ~/File-Center/Control, only after and this way it is built as it should.