Found the issue. Qt Creator wants you to use a compiler listed under one of their Qt libraries. Use the Maintenance Tool to install this.
To do so:
Go to Tools -> Options.... Select Build & Run on left. Open Kits tab. You should have Manual -> Desktop (default) line in list. Choose it. Now select something like Qt 5.5.1 in PATH (qt5) in Qt version combobox and click Apply button. From now you should be able to create, build and run empty Qt project.