One of the easiest way to do it is by using
npx --ignore-existing create-react-app [project name]
This will remove the old cached version of create-react-app and then get the new version to create the project.
Note: Adding the name of the project is important as just ignoring the existing create-react-app version is stale and the changes in your machines global env is temporary and hence later just using npx create-react-app [project name]
will not provide the desired result.