Are lists thread-safe

Question

I notice that it is often suggested to use queues with multiple threads  instead of lists and  pop    Is this because lists are not thread-safe  or for some other reason

User · Accepted Answer

Lists themselves are thread-safe  In CPython the GIL protects against concurrent accesses to them  and other implementations take care to use a fine-grained lock or a synchronized datatype for their list implementations  However  while lists themselves can t go corrupt by attempts to concurrently access  the lists s data is not protected  For example   L 0     1   is not guaranteed to actually increase L 0  by one if another thread does the same thing  because    is not an atomic operation   Very  very few operations in Python are actually atomic  because most of them can cause arbitrary Python code to be called   You should use Queues because if you just use an unprotected list  you may get or delete the wrong item because of race conditions

User · Answer

To clarify a point in Thomas  excellent answer  it should be mentioned that append   is thread safe   This is because there is no concern that data being read will be in the same place once we go to write to it  The append   operation does not read data  it only writes data to the list

User · Answer

I recently had this case where I needed to append to a list continuously in one thread  loop through the items and check if the item was ready  it was an AsyncResult in my case and remove it from the list only if it was ready    I could not find any examples that demonstrated my problem clearly  Here is an example demonstrating adding to list in one thread continuously and removing from the same list in another thread continuously  The flawed version runs easily on smaller numbers but keep the numbers big enough and run a few times and you will see the error  The FLAWED version  import threading import time    Change this number as you please  bigger numbers will get the error quickly count   1000 l       def add        for i in range count           l append i          time sleep 0 0001   def remove        for i in range count           l remove i          time sleep 0 0001    t1   threading Thread target add  t2   threading Thread target remove  t1 start   t2 start   t1 join   t2 join    print l    Output when ERROR  Exception in thread Thread-63  Traceback  most recent call last     File   Users zup  pyenv versions 3 6 8 lib python3 6 threading py   line 916  in  bootstrap inner     self run     File   Users zup  pyenv versions 3 6 8 lib python3 6 threading py   line 864  in run     self  target  self  args    self  kwargs    File   lt ipython-input-30-ecfbac1c776f gt    line 13  in remove     l remove i  ValueError  list remove x   x not in list   Version that uses locks  import threading import time count   1000 l      lock   threading RLock   def add        with lock          for i in range count               l append i              time sleep 0 0001   def remove        with lock          for i in range count               l remove i              time sleep 0 0001    t1   threading Thread target add  t2   threading Thread target remove  t1 start   t2 start   t1 join   t2 join    print l    Output       Empty list   Conclusion  As mentioned in the earlier answers while the act of appending or popping elements from the list itself is thread safe  what is not thread safe is when you append in one thread and pop in another

User · Answer

Here s a comprehensive yet non-exhaustive list of examples of list operations and whether or not they are thread safe  Hoping to get an answer regarding the obj in a list language construct here

[python] Are lists thread-safe?

Examples related to python

Examples related to multithreading

Examples related to list

Examples related to python-3.x

Examples related to python-multithreading