Factory classes are useful for when the object type that they return has a private constructor, when different factory classes set different properties on the returning object, or when a specific factory type is coupled with its returning concrete type.
WCF uses ServiceHostFactory classes to retrieve ServiceHost objects in different situations. The standard ServiceHostFactory is used by IIS to retrieve ServiceHost instances for .svc files, but a WebScriptServiceHostFactory is used for services that return serializations to JavaScript clients. ADO.NET Data Services has its own special DataServiceHostFactory and ASP.NET has its ApplicationServicesHostFactory since its services have private constructors.
If you only have one class that's consuming the factory, then you can just use a factory method within that class.