I had a similar issue when attempting to switch to from OutOfProcess hosting to InProcess hosting on a .Net Core project which I had recently upgraded from 2.0 to 3.0.
With no real helpful error to go on and after spending days trying to resolve this, I eventually found a fix for my case which I thought I'd share in case it is helpful to anyone else struggling with this.
For me, it was caused by a few Microsoft.AspNetCore packages.
After removing all of the referenced Microsoft.AspNetCore packages that had version less than 3.0.0 (there was no upgrade available >= 3.0.0 for these) this error no longer occurred.
These were the packages I removed;
<PackageReference Include="Microsoft.AspNetCore" Version="2.2.0" />
<PackageReference Include="Microsoft.AspNetCore.App" Version="2.2.8" />
<PackageReference Include="Microsoft.AspNetCore.Server.IISIntegration" Version="2.2.1" />
<PackageReference Include="Microsoft.AspNetCore.StaticFiles" Version="2.2.0" />
All other Microsoft.AspNetCore packages with version greater than or equal to 3.0.0 worked fine.