Just found a post Atomic vs. Non-Atomic Operations to be very helpful to me.
"An operation acting on shared memory is atomic if it completes in a single step relative to other threads.
When an atomic store is performed on a shared memory, no other thread can observe the modification half-complete.
When an atomic load is performed on a shared variable, it reads the entire value as it appeared at a single moment in time."