Instead of only using the path to your script in the task scheduler, you should start PowerShell with your script in the task scheduler, e.g.
C:\WINDOWS\system32\WindowsPowerShell\v1.0\powershell.exe -NoLogo -NonInteractive -File "C:\Path\To\Your\PS1File.ps1"
See powershell /?
for an explanation of those switches.
If you still get problems you should read this question.