A comparison between the different Visual Studio Express editions can be found at Visual Studio Express (archive.org link). The difference between Windows and Windows Desktop is that with the Windows edition you can build Windows Store Apps (using .NET, WPF/XAML) while the Windows Desktop edition allows you to write classic Windows Desktop applications. It is possible to install both products on the same machine.
Visual Studio Express 2010 allows you to build Windows Desktop applications. Writing Windows Store applications is not possible with this product.
For learning I would suggest Notepad and the command line. While an IDE provides significant productivity enhancements to professionals, it can be intimidating to a beginner. If you want to use an IDE nevertheless I would recommend Visual Studio Express 2013 for Windows Desktop.