Try typing all of this on the command line:
sqlplus / as sysdba
As what you are doing is starting sqlplus and then using sys as sysdba
as the user-name which is incorrect as that is not a valid user. By using the above command Oracle is using your system login credentials to access the db.
Also, I would confirm that the sqlplus executable you are running is the correct one by checking your path - ensure it is in the bin of the server installation directories.