As many of the members suggested use LinkedHashSet to retain the order of the collection. U can wrap your set using this implementation.
SortedSet implementation can be used for sorted order but for your purpose use LinkedHashSet.
Also from the docs,
"This implementation spares its clients from the unspecified, generally chaotic ordering provided by HashSet, without incurring the increased cost associated with TreeSet. It can be used to produce a copy of a set that has the same order as the original, regardless of the original set's implementation:"
Source : http://docs.oracle.com/javase/6/docs/api/java/util/LinkedHashSet.html