Non-alphanumeric list order from os listdir

Question

I often use python to process directories of data  Recently  I have noticed that the default order of the lists has changed to something almost nonsensical  For example  if I am in a current directory containing the following subdirectories  run01  run02      run19  run20  and then I generate a list from the following command   dir   os listdir os getcwd      then I usually get a list in this order   dir     run01    run18    run14    run13    run12    run11    run08           and so on  The order used to be alphanumeric  But this new order has remained with me for a while now   What is determining the  displayed  order of these lists

User · Answer

From the documentation      The list is in arbitrary order  and does not include the special entries     and      even if they are present in the directory    This means that the order is probably OS filesystem dependent  has no particularly meaningful order  and is therefore not guaranteed to be anything in particular  As many answers mentioned  if preferred  the retrieved list can be sorted   Cheers

User · Answer

I think the order has to do with the way the files are indexed on your FileSystem  If you really want to make it adhere to some order you can always sort the list after getting the files

User · Answer

It s probably just the order that C s readdir   returns   Try running this C program    include  lt dirent h gt   include  lt stdio h gt  int main void      DIR  dirp      struct dirent  de      dirp   opendir           while de   readdir dirp      Yes  one              printf   s n   de- gt d name       closedir dirp       return 0      The build line should be something like gcc -o foo foo c   P S  Just ran this and your Python code  and they both gave me sorted output  so I can t reproduce what you re seeing

User · Answer

Python for whatever reason does not come with a built-in way to have natural sorting  meaning 1  2  10 instead of 1  10  2   so you have to write it yourself  import re def sorted alphanumeric data       convert   lambda text  int text  if text isdigit   else text lower       alphanum key   lambda key    convert c  for c in re split    0-9      key         return sorted data  key alphanum key   You can now use this function to sort a list  dirlist   sorted alphanumeric os listdir         PROBLEMS  In case you use the above function to sort strings  for example folder names  and want them sorted like Windows Explorer does  it will not work properly in some edge cases  This sorting function will return incorrect results on Windows  if you have folder names with certain  special  characters in them  For example this function will sort 1   1   a  a  whereas Windows Explorer would sort  1  1   a  a  So if you want to sort exactly like Windows Explorer does in Python you have to use the Windows built-in function StrCmpLogicalW via ctypes  this of course won t work on Unix   from ctypes import wintypes  windll from functools import cmp to key  def winsort data        StrCmpLogicalW   windll Shlwapi StrCmpLogicalW      StrCmpLogicalW argtypes    wintypes LPWSTR  wintypes LPWSTR       StrCmpLogicalW restype    wintypes INT      cmp fnc   lambda psz1  psz2   StrCmpLogicalW psz1  psz2      return sorted data  key cmp to key cmp fnc    This function is slightly slower than sorted alphanumeric    Bonus  winsort can also sort full paths on Windows  Alternatively  especially if you use Unix  you can use the natsort library  pip install natsort  to sort by full paths in a correct way  meaning subfolders at the correct position   You can use it like this to sort full paths  from natsort import natsorted  ns dirlist   natsorted dirlist  alg ns PATH   ns IGNORECASE   Starting with version 7 1 0 natsort supports os sorted which internally uses either the beforementioned Windows API or Linux sorting and should be used instead of natsorted

User · Answer

In  6   os listdir   Type        builtin function or method String Form  lt built-in function listdir gt  Docstring  listdir path  - gt  list of strings Return a list containing the names of the entries in the directory  path  path of directory to list The list is in   arbitrary order     It does not include the special entries     and      even if they are present in the directory

User · Answer

You can use the builtin sorted function to sort the strings however you want   Based on what you describe    sorted os listdir whatever directory     Alternatively  you can use the  sort method of a list   lst   os listdir whatever directory  lst sort     I think should do the trick   Note that the order that os listdir gets the filenames is probably completely dependent on your filesystem

User · Answer

aaa     row 163 pkl    row 394 pkl    row 679 pkl    row 202 pkl    row 1449 pkl    row 247 pkl    row 1353 pkl    row 749 pkl    row 1293 pkl    row 1304 pkl    row 78 pkl    row 532 pkl    row 9 pkl    row 1435 pkl                                                                                                                                                                                                                                                                                                    sorted aaa  key lambda x  int os path splitext x split      1   0      As In case of mine requirement I have the case like row 163 pkl here os path splitext  row 163 pkl   will break it into   row 163     pkl   so need to split it based on     also   but in case of your requirement you can do something like  sorted aa  key   lambda x   int re sub   D     x   x     where   aa     run01    run08    run11    run12    run13    run14    run18     and also for directory retrieving you can do sorted os listdir path    and for the case of like  run01 txt  or  run01 csv  you can do like this   sorted files  key lambda x   int os path splitext x  0

User · Answer

The proposed combination of os listdir and sorted commands generates the same result as ls -l command under Linux  The following example verifies this assumption  user user-PC  tmp test  touch 3a 4a 5a b c d1 d2 d3 k l p0 p1 p3 q 410a 409a 408a 407a user user-PC  tmp test  ls -l total 0 -rw-rw-r-- 1 user user 0 Feb  15 10 31 3a -rw-rw-r-- 1 user user 0 Feb  15 10 31 407a -rw-rw-r-- 1 user user 0 Feb  15 10 31 408a -rw-rw-r-- 1 user user 0 Feb  15 10 31 409a -rw-rw-r-- 1 user user 0 Feb  15 10 31 410a -rw-rw-r-- 1 user user 0 Feb  15 10 31 4a -rw-rw-r-- 1 user user 0 Feb  15 10 31 5a -rw-rw-r-- 1 user user 0 Feb  15 10 31 b -rw-rw-r-- 1 user user 0 Feb  15 10 31 c -rw-rw-r-- 1 user user 0 Feb  15 10 31 d1 -rw-rw-r-- 1 user user 0 Feb  15 10 31 d2 -rw-rw-r-- 1 user user 0 Feb  15 10 31 d3 -rw-rw-r-- 1 user user 0 Feb  15 10 31 k -rw-rw-r-- 1 user user 0 Feb  15 10 31 l -rw-rw-r-- 1 user user 0 Feb  15 10 31 p0 -rw-rw-r-- 1 user user 0 Feb  15 10 31 p1 -rw-rw-r-- 1 user user 0 Feb  15 10 31 p3 -rw-rw-r-- 1 user user 0 Feb  15 10 31 q  user user-PC  tmp test  python Python 2 7 6  default  Jun 22 2015  17 58 13    GCC 4 8 2  on linux2 Type  quot help quot    quot copyright quot    quot credits quot  or  quot license quot  for more information   gt  gt  gt  import os  gt  gt  gt  os listdir           d3    k    p1    b    410a    5a    l    p0    407a    409a    408a    d2    4a    p3    3a    q    c    d1    gt  gt  gt  sorted  os listdir             3a    407a    408a    409a    410a    4a    5a    b    c    d1    d2    d3    k    l    p0    p1    p3    q    gt  gt  gt  exit   user user-PC  tmp test    So  for someone who wants to reproduce the result of the well-known ls -l command in their python code  sorted  os listdir  DIR     works pretty well

User · Answer

ls by default previews the files sorted by name   ls options can be used to sort by date  size       files   list os popen  quot ls quot    files    file strip  quot  n quot   for file in files   Using ls would have much better performance when the directory contains so many files

User · Answer

I found  sort  does not always do what I expected  eg  I have a directory as below  and the  sort  give me a very strange result    gt  gt  gt  os listdir pathon    2    3    4    5    403    404    407    408    410    411    412    413    414    415    416    472    gt  gt  gt  sorted   f for f in os listdir pathon      2    3    4    403    404    407    408    410    411    412    413    414    415    416    472    5     It seems it compares the first character first  if that is the biggest  it would be the last one

User · Answer

I think by default the order is determined with the ASCII value  The solution to this problem is this  dir   sorted os listdir os getcwd     key len

User · Answer

Use natsort library  Install the library with the following command for Ubuntu and other Debian versions Python 2 sudo pip install natsort  Python 3 sudo pip3 install natsort  Details of how to use this library is found here from natsort import natsorted  files     run01    run18    run14    run13    run12    run11    run08   natsorted files    out     run01    run08    run11    run12    run13    run14    run18     This is not a duplicate of answer  natsort was added as an edit on 2020-01-27

User · Answer

Per the documentation      os listdir path              Return a list     containing the names of the entries in     the directory given by path  The list     is in arbitrary order  It does not     include the special entries     and          even if they are present in the     directory       Order cannot be relied upon and is an artifact of the filesystem   To sort the result  use sorted os listdir path

[python] Non-alphanumeric list order from os.listdir()

Examples related to python

Examples related to list

Examples related to directory-listing

Examples related to listdir