Your CAST() looks correct.
Your CONVERT() is not correct. You are quoting the column as a string. You will want something like
CONVERT(INT, A.my_NvarcharColumn)
** notice without the quotes **
The only other reason why this could fail is if you have a non-numeric character in the field value or if it's out of range.
You can try something like the following to verify it's numeric and return a NULL if it's not:
SELECT
CASE
WHEN ISNUMERIC(A.my_NvarcharColumn) = 1 THEN CONVERT(INT, A.my_NvarcharColumn)
ELSE NULL
END AS my_NvarcharColumn