The volatile and transient modifiers can be applied to fields of classes1 irrespective of field type. Apart from that, they are unrelated.
The transient modifier tells the Java object serialization subsystem to exclude the field when serializing an instance of the class. When the object is then deserialized, the field will be initialized to the default value; i.e. null for a reference type, and zero or false for a primitive type. Note that the JLS (see 8.3.1.3) does not say what transient means, but defers to the Java Object Serialization Specification. Other serialization mechanisms may pay attention to a field's transient-ness. Or they may ignore it.
(Note that the JLS permits a static field to be declared as transient. This combination doesn't make sense for Java Object Serialization, since it doesn't serialize statics anyway. However, it could make sense in other contexts, so there is some justification for not forbidding it outright.)
The volatile modifier tells the JVM that writes to the field should always be synchronously flushed to memory, and that reads of the field should always read from memory. This means that fields marked as volatile can be safely accessed and updated in a multi-thread application without using native or standard library-based synchronization. Similarly, reads and writes to volatile fields are atomic. (This does not apply to >>non-volatile<< long or double fields, which may be subject to "word tearing" on some JVMs.) The relevant parts of the JLS are 8.3.1.4, 17.4 and 17.7.
1 - But not to local variables or parameters.