How do I watch a file for changes

Question

I have a log file being written by another process which I want to watch for changes  Each time a change occurs I d like to read the new data in to do some processing on it   What s the best way to do this  I was hoping there d be some sort of hook from the PyWin32 library  I ve found the win32file FindNextChangeNotification function but have no idea how to ask it to watch a specific file   If anyone s done anything like this I d be really grateful to hear how      Edit  I should have mentioned that I was after a solution that doesn t require polling    Edit  Curses  It seems this doesn t work over a mapped network drive  I m guessing windows doesn t  hear  any updates to the file the way it does on a local disk

User · Answer

As you can see in Tim Golden s article  pointed by Horst Gutmann  WIN32 is relatively complex and watches directories  not a single file   I d like to suggest you look into IronPython  which is a  NET python implementation  With IronPython you can use all the  NET functionality - including  System IO FileSystemWatcher   Which handles single files with a simple Event interface

User · Answer

Check my answer to a similar question  You could try the same loop in Python  This page suggests   import time  while 1      where   file tell       line   file readline       if not line          time sleep 1          file seek where      else          print line    already has newline   Also see the question tail   a file with Python

User · Answer

Well  since you are using Python  you can just open a file and keep reading lines from it   f   open  file log     If the line read is not empty  you process it   line   f readline   if line         Do what you want with the line   You may be missing that it is ok to keep calling readline at the EOF  It will just keep returning an empty string in this case  And when something is appended to the log file  the reading will continue from where it stopped  as you need   If you are looking for a solution that uses events  or a particular library  please specify this in your question  Otherwise  I think this solution is just fine

User · Answer

If polling is good enough for you  I d just watch if the  modified time  file stat changes   To read it   os stat filename  st mtime    Also note that the Windows native change event solution does not work in all circumstances  e g  on network drives    import os  class Monkey object       def   init   self           self  cached stamp   0         self filename     path to file       def ook self           stamp   os stat self filename  st mtime         if stamp    self  cached stamp              self  cached stamp   stamp               File has changed  so do something

User · Answer

This is another modification of Tim Goldan s script that runs on unix types and adds a simple watcher for file modification by using a dict  file  time    usage  whateverName py path to dir to watch     usr bin env python  import os  sys  time  def files to timestamp path       files    os path join path  f  for f in os listdir path       return dict    f  os path getmtime f   for f in files    if   name         main          path to watch   sys argv 1      print  Watching       format path to watch        before   files to timestamp path to watch       while 1          time sleep  2          after   files to timestamp path to watch           added    f for f in after keys   if not f in before keys            removed    f for f in before keys   if not f in after keys            modified               for f in before keys                if not f in removed                  if os path getmtime f     before get f                       modified append f           if added  print  Added      format      join added            if removed  print  Removed      format      join removed            if modified  print  Modified      format      join modified             before   after

User · Answer

This is an example of checking a file for changes  One that may not be the best way of doing it  but it sure is a short way   Handy tool for restarting application when changes have been made to the source  I made this when playing with pygame so I can see effects take place immediately after file save   When used in pygame make sure the stuff in the  while  loop is placed in your game loop aka update or whatever  Otherwise your application will get stuck in an infinite loop and you will not see your game updating   file size stored   os stat  neuron py   st size    while True      try        file size current   os stat  neuron py   st size       if file size stored    file size current          restart program       except         pass   In case you wanted the restart code which I found on the web  Here it is   Not relevant to the question  though it could come in handy   def restart program     restart application     python   sys executable     os execl python  python    sys argv    Have fun making electrons do what you want them to do

User · Answer

If you want a multiplatform solution  then check QFileSystemWatcher  Here an example code  not sanitized    from PyQt4 import QtCore   QtCore pyqtSlot str  def directory changed path       print  Directory Changed        QtCore pyqtSlot str  def file changed path       print  File Changed       fs watcher   QtCore QFileSystemWatcher    path to files 1     path to files 2     path to files 3     fs watcher connect fs watcher  QtCore SIGNAL  directoryChanged QString     directory changed  fs watcher connect fs watcher  QtCore SIGNAL  fileChanged QString     file changed

User · Answer

Well after a bit of hacking of Tim Golden s script  I have the following which seems to work quite well   import os  import win32file import win32con  path to watch         look at the current directory file to watch    test txt    look for changes to a file called test txt  def ProcessNewData  newData        print  Text added   s  newData    Set up the bits we ll need for output ACTIONS       1    Created     2    Deleted     3    Updated     4    Renamed from something     5    Renamed to something    FILE LIST DIRECTORY   0x0001 hDir   win32file CreateFile     path to watch    FILE LIST DIRECTORY    win32con FILE SHARE READ   win32con FILE SHARE WRITE    None    win32con OPEN EXISTING    win32con FILE FLAG BACKUP SEMANTICS    None      Open the file we re interested in a   open file to watch   r      Throw away any exising log data a read      Wait for new data and call ProcessNewData for each new chunk that s written while 1      Wait for a change to occur   results   win32file ReadDirectoryChangesW       hDir      1024      False      win32con FILE NOTIFY CHANGE LAST WRITE      None      None          For each change  check to see if it s updating the file we re interested in   for action  file in results      full filename   os path join  path to watch  file       print file  ACTIONS get  action   Unknown       if file    file to watch          newText   a read           if newText                    ProcessNewData  newText     It could probably do with a load more error checking  but for simply watching a log file and doing some processing on it before spitting it out to the screen  this works well   Thanks everyone for your input - great stuff

User · Answer

If you re using windows  create this POLL CMD file  echo off  top xcopy  m  y  1  2   find  v  quot File s  copied quot  timeout  T 1  gt  nul goto  top  then you can type  quot poll dir1 dir2 quot  and it will copy all the files from dir1 to dir2 and check for updates once per second  The  quot find quot  is optional  just to make the console less noisy  This is not recursive  Maybe you could make it recursive using  e on the xcopy

User · Answer

Check my answer to a similar question  You could try the same loop in Python  This page suggests   import time  while 1      where   file tell       line   file readline       if not line          time sleep 1          file seek where      else          print line    already has newline   Also see the question tail   a file with Python

User · Answer

Here s an example geared toward watching input files that write no more than one line per second but usually a lot less   The goal is to append the last line  most recent write  to the specified output file   I ve copied this from one of my projects and just deleted all the irrelevant lines   You ll have to fill in or change the missing symbols    from PyQt5 QtCore import QFileSystemWatcher  QSettings  QThread from ui main window import Ui MainWindow     Qt Creator gen d   class MainWindow QMainWindow  Ui MainWindow       def   init   self  parent None           QMainWindow   init   self  parent          Ui MainWindow   init   self          self  fileWatcher   QFileSystemWatcher           self  fileWatcher fileChanged connect self fileChanged       def fileChanged self  filepath           QThread msleep 300       Reqd on some machines  give chance for write to complete              About to test this  may need more sophisticated solution         with open filepath  as file              lastLine   list file  -1          destPath   self  filemap filepath   dest file           with open destPath   a   as out file                  a  append             out file writelines  lastLine     Of course  the encompassing QMainWindow class is not strictly required  ie  you can use QFileSystemWatcher alone

User · Answer

For watching a single file with polling  and minimal dependencies  here is a fully fleshed-out example  based on answer from Deestan  above    import os import sys  import time  class Watcher object       running   True     refresh delay secs   1        Constructor     def   init   self  watch file  call func on change None   args    kwargs           self  cached stamp   0         self filename   watch file         self call func on change   call func on change         self args   args         self kwargs   kwargs        Look for changes     def look self           stamp   os stat self filename  st mtime         if stamp    self  cached stamp              self  cached stamp   stamp               File has changed  so do something                print  File changed               if self call func on change is not None                  self call func on change  self args    self kwargs         Keep watching in a loop             def watch self           while self running               try                     Look for changes                 time sleep self refresh delay secs                   self look                except KeyboardInterrupt                   print   nDone                    break              except FileNotFoundError                    Action on file not found                 pass             except                   print  Unhandled error   s    sys exc info   0      Call this function each time a change happens def custom action text       print text   watch file    my file txt     watcher   Watcher watch file     simple watcher   Watcher watch file  custom action  text  yes  changed      also call custom action function watcher watch      start the watch going

User · Answer

Well  since you are using Python  you can just open a file and keep reading lines from it   f   open  file log     If the line read is not empty  you process it   line   f readline   if line         Do what you want with the line   You may be missing that it is ok to keep calling readline at the EOF  It will just keep returning an empty string in this case  And when something is appended to the log file  the reading will continue from where it stopped  as you need   If you are looking for a solution that uses events  or a particular library  please specify this in your question  Otherwise  I think this solution is just fine

User · Answer

If polling is good enough for you  I d just watch if the  modified time  file stat changes   To read it   os stat filename  st mtime    Also note that the Windows native change event solution does not work in all circumstances  e g  on network drives    import os  class Monkey object       def   init   self           self  cached stamp   0         self filename     path to file       def ook self           stamp   os stat self filename  st mtime         if stamp    self  cached stamp              self  cached stamp   stamp               File has changed  so do something

User · Answer

ACTIONS       1    Created     2    Deleted     3    Updated     4    Renamed from something     5    Renamed to something    FILE LIST DIRECTORY   0x0001  class myThread  threading Thread       def   init   self  threadID  fileName  directory  origin           threading Thread   init   self          self threadID   threadID         self fileName   fileName         self daemon   True         self dir   directory         self originalFile   origin     def run self           startMonitor self fileName  self dir  self originalFile   def startMonitor fileMonitoring dirPath originalFile       hDir   win32file CreateFile           dirPath          FILE LIST DIRECTORY          win32con FILE SHARE READ   win32con FILE SHARE WRITE          None          win32con OPEN EXISTING          win32con FILE FLAG BACKUP SEMANTICS          None             Wait for new data and call ProcessNewData for each new chunk that s       written     while 1            Wait for a change to occur         results   win32file ReadDirectoryChangesW               hDir              1024              False              win32con FILE NOTIFY CHANGE LAST WRITE              None              None                     For each change  check to see if it s updating the file we re           interested in         for action  file M in results              full filename   os path join  dirPath  file M               print file  ACTIONS get  action   Unknown               if len full filename     len fileMonitoring  and action    3                   copy to main file

User · Answer

Have you already looked at the documentation available on http   timgolden me uk python win32 how do i watch directory for changes html  If you only need it to work under Windows the 2nd example seems to be exactly what you want  if you exchange the path of the directory with one of the files you want to watch   Otherwise  polling will probably be the only really platform-independent option  Note  I haven t tried any of these solutions

User · Answer

If you want a multiplatform solution  then check QFileSystemWatcher  Here an example code  not sanitized    from PyQt4 import QtCore   QtCore pyqtSlot str  def directory changed path       print  Directory Changed        QtCore pyqtSlot str  def file changed path       print  File Changed       fs watcher   QtCore QFileSystemWatcher    path to files 1     path to files 2     path to files 3     fs watcher connect fs watcher  QtCore SIGNAL  directoryChanged QString     directory changed  fs watcher connect fs watcher  QtCore SIGNAL  fileChanged QString     file changed

User · Answer

Well  since you are using Python  you can just open a file and keep reading lines from it   f   open  file log     If the line read is not empty  you process it   line   f readline   if line         Do what you want with the line   You may be missing that it is ok to keep calling readline at the EOF  It will just keep returning an empty string in this case  And when something is appended to the log file  the reading will continue from where it stopped  as you need   If you are looking for a solution that uses events  or a particular library  please specify this in your question  Otherwise  I think this solution is just fine

User · Answer

I don t know any Windows specific function  You could try getting the MD5 hash of the file every second minute hour  depends on how fast you need it  and compare it to the last hash  When it differs you know the file has been changed and you read out the newest lines

User · Answer

I don t know any Windows specific function  You could try getting the MD5 hash of the file every second minute hour  depends on how fast you need it  and compare it to the last hash  When it differs you know the file has been changed and you read out the newest lines

User · Answer

Simplest solution for me is using watchdog s tool watchmedo  From https   pypi python org pypi watchdog I now have a process that looks up the sql files in a directory and executes them if necessary    watchmedo shell-command   --patterns    sql    --recursive   --command    Desktop load files into mysql database sh

User · Answer

I don t know any Windows specific function  You could try getting the MD5 hash of the file every second minute hour  depends on how fast you need it  and compare it to the last hash  When it differs you know the file has been changed and you read out the newest lines

User · Answer

As you can see in Tim Golden s article  pointed by Horst Gutmann  WIN32 is relatively complex and watches directories  not a single file   I d like to suggest you look into IronPython  which is a  NET python implementation  With IronPython you can use all the  NET functionality - including  System IO FileSystemWatcher   Which handles single files with a simple Event interface

User · Answer

The best and simplest solution is to use pygtail     https   pypi python org pypi pygtail  from pygtail import Pygtail import sys  while True      for line in Pygtail  some log            sys stdout write line

User · Answer

Here is a simplified version of Kender s code that appears to do the same trick and does not import the entire file     Check file for new data   import time  f   open r c  temp test txt    r    while True       line   f readline       if not line          time sleep 1          print  Nothing New      else          print  Call Function     line

User · Answer

It should not work on windows  maybe with cygwin     but for unix user  you should use the  fcntl  system call  Here is an example in Python  It s mostly the same code if you need to write it in C  same function names   import time import fcntl import os import signal  FNAME     HOME TOTO FILETOWATCH   def handler signum  frame       print  File  s modified     FNAME    signal signal signal SIGIO  handler  fd   os open FNAME   os O RDONLY  fcntl fcntl fd  fcntl F SETSIG  0  fcntl fcntl fd  fcntl F NOTIFY              fcntl DN MODIFY   fcntl DN CREATE   fcntl DN MULTISHOT   while True      time sleep 10000

User · Answer

Since I have it installed globally  my favorite approach is to use nodemon  If your source code is in src  and your entry point is src app py  then it s as easy as  nodemon -w  src     -e py html --exec python src app py      where -e py html lets you control what file types to watch for changes

User · Answer

Well after a bit of hacking of Tim Golden s script  I have the following which seems to work quite well   import os  import win32file import win32con  path to watch         look at the current directory file to watch    test txt    look for changes to a file called test txt  def ProcessNewData  newData        print  Text added   s  newData    Set up the bits we ll need for output ACTIONS       1    Created     2    Deleted     3    Updated     4    Renamed from something     5    Renamed to something    FILE LIST DIRECTORY   0x0001 hDir   win32file CreateFile     path to watch    FILE LIST DIRECTORY    win32con FILE SHARE READ   win32con FILE SHARE WRITE    None    win32con OPEN EXISTING    win32con FILE FLAG BACKUP SEMANTICS    None      Open the file we re interested in a   open file to watch   r      Throw away any exising log data a read      Wait for new data and call ProcessNewData for each new chunk that s written while 1      Wait for a change to occur   results   win32file ReadDirectoryChangesW       hDir      1024      False      win32con FILE NOTIFY CHANGE LAST WRITE      None      None          For each change  check to see if it s updating the file we re interested in   for action  file in results      full filename   os path join  path to watch  file       print file  ACTIONS get  action   Unknown       if file    file to watch          newText   a read           if newText                    ProcessNewData  newText     It could probably do with a load more error checking  but for simply watching a log file and doing some processing on it before spitting it out to the screen  this works well   Thanks everyone for your input - great stuff

User · Answer

If you re using windows  create this POLL CMD file  echo off  top xcopy  m  y  1  2   find  v  quot File s  copied quot  timeout  T 1  gt  nul goto  top  then you can type  quot poll dir1 dir2 quot  and it will copy all the files from dir1 to dir2 and check for updates once per second  The  quot find quot  is optional  just to make the console less noisy  This is not recursive  Maybe you could make it recursive using  e on the xcopy

User · Answer

Seems that no one has posted fswatch  It is a cross-platform file system watcher  Just install it  run it and follow the prompts   I ve used it with python and golang programs and it just works

User · Answer

I d try something like this       try              f   open filePath      except IOError              print  No such file   s    filePath             raw input  Press Enter to close window       try              lines   f readlines               while True                      line   f readline                       try                              if not line                                      time sleep 1                              else                                      functionThatAnalisesTheLine line                      except Exception  e                                handle the exception somehow  for example  log the trace  and raise the same exception again                             raw input  Press Enter to close window                               raise e     finally              f close     The loop checks if there is a new line s  since last time file was read - if there is  it s read and passed to the functionThatAnalisesTheLine function  If not  script waits 1 second and retries the process

User · Answer

I d try something like this       try              f   open filePath      except IOError              print  No such file   s    filePath             raw input  Press Enter to close window       try              lines   f readlines               while True                      line   f readline                       try                              if not line                                      time sleep 1                              else                                      functionThatAnalisesTheLine line                      except Exception  e                                handle the exception somehow  for example  log the trace  and raise the same exception again                             raw input  Press Enter to close window                               raise e     finally              f close     The loop checks if there is a new line s  since last time file was read - if there is  it s read and passed to the functionThatAnalisesTheLine function  If not  script waits 1 second and retries the process

User · Answer

ACTIONS       1    Created     2    Deleted     3    Updated     4    Renamed from something     5    Renamed to something    FILE LIST DIRECTORY   0x0001  class myThread  threading Thread       def   init   self  threadID  fileName  directory  origin           threading Thread   init   self          self threadID   threadID         self fileName   fileName         self daemon   True         self dir   directory         self originalFile   origin     def run self           startMonitor self fileName  self dir  self originalFile   def startMonitor fileMonitoring dirPath originalFile       hDir   win32file CreateFile           dirPath          FILE LIST DIRECTORY          win32con FILE SHARE READ   win32con FILE SHARE WRITE          None          win32con OPEN EXISTING          win32con FILE FLAG BACKUP SEMANTICS          None             Wait for new data and call ProcessNewData for each new chunk that s       written     while 1            Wait for a change to occur         results   win32file ReadDirectoryChangesW               hDir              1024              False              win32con FILE NOTIFY CHANGE LAST WRITE              None              None                     For each change  check to see if it s updating the file we re           interested in         for action  file M in results              full filename   os path join  dirPath  file M               print file  ACTIONS get  action   Unknown               if len full filename     len fileMonitoring  and action    3                   copy to main file

User · Answer

The best and simplest solution is to use pygtail     https   pypi python org pypi pygtail  from pygtail import Pygtail import sys  while True      for line in Pygtail  some log            sys stdout write line

User · Answer

Well after a bit of hacking of Tim Golden s script  I have the following which seems to work quite well   import os  import win32file import win32con  path to watch         look at the current directory file to watch    test txt    look for changes to a file called test txt  def ProcessNewData  newData        print  Text added   s  newData    Set up the bits we ll need for output ACTIONS       1    Created     2    Deleted     3    Updated     4    Renamed from something     5    Renamed to something    FILE LIST DIRECTORY   0x0001 hDir   win32file CreateFile     path to watch    FILE LIST DIRECTORY    win32con FILE SHARE READ   win32con FILE SHARE WRITE    None    win32con OPEN EXISTING    win32con FILE FLAG BACKUP SEMANTICS    None      Open the file we re interested in a   open file to watch   r      Throw away any exising log data a read      Wait for new data and call ProcessNewData for each new chunk that s written while 1      Wait for a change to occur   results   win32file ReadDirectoryChangesW       hDir      1024      False      win32con FILE NOTIFY CHANGE LAST WRITE      None      None          For each change  check to see if it s updating the file we re interested in   for action  file in results      full filename   os path join  path to watch  file       print file  ACTIONS get  action   Unknown       if file    file to watch          newText   a read           if newText                    ProcessNewData  newText     It could probably do with a load more error checking  but for simply watching a log file and doing some processing on it before spitting it out to the screen  this works well   Thanks everyone for your input - great stuff

User · Answer

Check my answer to a similar question  You could try the same loop in Python  This page suggests   import time  while 1      where   file tell       line   file readline       if not line          time sleep 1          file seek where      else          print line    already has newline   Also see the question tail   a file with Python

User · Answer

It should not work on windows  maybe with cygwin     but for unix user  you should use the  fcntl  system call  Here is an example in Python  It s mostly the same code if you need to write it in C  same function names   import time import fcntl import os import signal  FNAME     HOME TOTO FILETOWATCH   def handler signum  frame       print  File  s modified     FNAME    signal signal signal SIGIO  handler  fd   os open FNAME   os O RDONLY  fcntl fcntl fd  fcntl F SETSIG  0  fcntl fcntl fd  fcntl F NOTIFY              fcntl DN MODIFY   fcntl DN CREATE   fcntl DN MULTISHOT   while True      time sleep 10000

User · Answer

I d try something like this       try              f   open filePath      except IOError              print  No such file   s    filePath             raw input  Press Enter to close window       try              lines   f readlines               while True                      line   f readline                       try                              if not line                                      time sleep 1                              else                                      functionThatAnalisesTheLine line                      except Exception  e                                handle the exception somehow  for example  log the trace  and raise the same exception again                             raw input  Press Enter to close window                               raise e     finally              f close     The loop checks if there is a new line s  since last time file was read - if there is  it s read and passed to the functionThatAnalisesTheLine function  If not  script waits 1 second and retries the process

User · Answer

Have you already looked at the documentation available on http   timgolden me uk python win32 how do i watch directory for changes html  If you only need it to work under Windows the 2nd example seems to be exactly what you want  if you exchange the path of the directory with one of the files you want to watch   Otherwise  polling will probably be the only really platform-independent option  Note  I haven t tried any of these solutions

User · Answer

As you can see in Tim Golden s article  pointed by Horst Gutmann  WIN32 is relatively complex and watches directories  not a single file   I d like to suggest you look into IronPython  which is a  NET python implementation  With IronPython you can use all the  NET functionality - including  System IO FileSystemWatcher   Which handles single files with a simple Event interface

User · Answer

This is an example of checking a file for changes  One that may not be the best way of doing it  but it sure is a short way   Handy tool for restarting application when changes have been made to the source  I made this when playing with pygame so I can see effects take place immediately after file save   When used in pygame make sure the stuff in the  while  loop is placed in your game loop aka update or whatever  Otherwise your application will get stuck in an infinite loop and you will not see your game updating   file size stored   os stat  neuron py   st size    while True      try        file size current   os stat  neuron py   st size       if file size stored    file size current          restart program       except         pass   In case you wanted the restart code which I found on the web  Here it is   Not relevant to the question  though it could come in handy   def restart program     restart application     python   sys executable     os execl python  python    sys argv    Have fun making electrons do what you want them to do

User · Answer

It should not work on windows  maybe with cygwin     but for unix user  you should use the  fcntl  system call  Here is an example in Python  It s mostly the same code if you need to write it in C  same function names   import time import fcntl import os import signal  FNAME     HOME TOTO FILETOWATCH   def handler signum  frame       print  File  s modified     FNAME    signal signal signal SIGIO  handler  fd   os open FNAME   os O RDONLY  fcntl fcntl fd  fcntl F SETSIG  0  fcntl fcntl fd  fcntl F NOTIFY              fcntl DN MODIFY   fcntl DN CREATE   fcntl DN MULTISHOT   while True      time sleep 10000

User · Answer

As you can see in Tim Golden s article  pointed by Horst Gutmann  WIN32 is relatively complex and watches directories  not a single file   I d like to suggest you look into IronPython  which is a  NET python implementation  With IronPython you can use all the  NET functionality - including  System IO FileSystemWatcher   Which handles single files with a simple Event interface

User · Answer

Here is a simplified version of Kender s code that appears to do the same trick and does not import the entire file     Check file for new data   import time  f   open r c  temp test txt    r    while True       line   f readline       if not line          time sleep 1          print  Nothing New      else          print  Call Function     line

User · Answer

Since I have it installed globally  my favorite approach is to use nodemon  If your source code is in src  and your entry point is src app py  then it s as easy as  nodemon -w  src     -e py html --exec python src app py      where -e py html lets you control what file types to watch for changes

User · Answer

This is another modification of Tim Goldan s script that runs on unix types and adds a simple watcher for file modification by using a dict  file  time    usage  whateverName py path to dir to watch     usr bin env python  import os  sys  time  def files to timestamp path       files    os path join path  f  for f in os listdir path       return dict    f  os path getmtime f   for f in files    if   name         main          path to watch   sys argv 1      print  Watching       format path to watch        before   files to timestamp path to watch       while 1          time sleep  2          after   files to timestamp path to watch           added    f for f in after keys   if not f in before keys            removed    f for f in before keys   if not f in after keys            modified               for f in before keys                if not f in removed                  if os path getmtime f     before get f                       modified append f           if added  print  Added      format      join added            if removed  print  Removed      format      join removed            if modified  print  Modified      format      join modified             before   after

User · Answer

If polling is good enough for you  I d just watch if the  modified time  file stat changes   To read it   os stat filename  st mtime    Also note that the Windows native change event solution does not work in all circumstances  e g  on network drives    import os  class Monkey object       def   init   self           self  cached stamp   0         self filename     path to file       def ook self           stamp   os stat self filename  st mtime         if stamp    self  cached stamp              self  cached stamp   stamp               File has changed  so do something

User · Answer

Here s an example geared toward watching input files that write no more than one line per second but usually a lot less   The goal is to append the last line  most recent write  to the specified output file   I ve copied this from one of my projects and just deleted all the irrelevant lines   You ll have to fill in or change the missing symbols    from PyQt5 QtCore import QFileSystemWatcher  QSettings  QThread from ui main window import Ui MainWindow     Qt Creator gen d   class MainWindow QMainWindow  Ui MainWindow       def   init   self  parent None           QMainWindow   init   self  parent          Ui MainWindow   init   self          self  fileWatcher   QFileSystemWatcher           self  fileWatcher fileChanged connect self fileChanged       def fileChanged self  filepath           QThread msleep 300       Reqd on some machines  give chance for write to complete              About to test this  may need more sophisticated solution         with open filepath  as file              lastLine   list file  -1          destPath   self  filemap filepath   dest file           with open destPath   a   as out file                  a  append             out file writelines  lastLine     Of course  the encompassing QMainWindow class is not strictly required  ie  you can use QFileSystemWatcher alone

User · Answer

Have you already looked at the documentation available on http   timgolden me uk python win32 how do i watch directory for changes html  If you only need it to work under Windows the 2nd example seems to be exactly what you want  if you exchange the path of the directory with one of the files you want to watch   Otherwise  polling will probably be the only really platform-independent option  Note  I haven t tried any of these solutions

User · Answer

related  4Oh4 solution a smooth change for a list of files to watch   import os import sys import time  class Watcher object       running   True     refresh delay secs   1        Constructor     def   init   self  watch files  call func on change None   args    kwargs           self  cached stamp   0         self  cached stamp files              self filenames   watch files         self call func on change   call func on change         self args   args         self kwargs   kwargs        Look for changes     def look self           for file in self filenames              stamp   os stat file  st mtime             if not file in self  cached stamp files                  self  cached stamp files file    0             if stamp    self  cached stamp files file                   self  cached stamp files file    stamp                   File has changed  so do something                    file to read   open file   r                   value   file to read read                   print  value from file   value                  file to read seek 0                  if self call func on change is not None                      self call func on change  self args    self kwargs         Keep watching in a loop     def watch self           while self running              try                    Look for changes                 time sleep self refresh delay secs                  self look               except KeyboardInterrupt                  print   nDone                   break             except FileNotFoundError                    Action on file not found                 pass             except Exception as e                  print e                  print  Unhandled error   s    sys exc info   0      Call this function each time a change happens def custom action text       print text        pass  watch files      Users mexekanez my file txt     Users mexekanez my file1 txt      watcher   Watcher watch file     simple    if   name         main         watcher   Watcher watch files  custom action  text  yes  changed      also call custom action function     watcher watch      start the watch going

User · Answer

Seems that no one has posted fswatch  It is a cross-platform file system watcher  Just install it  run it and follow the prompts   I ve used it with python and golang programs and it just works

User · Answer

Did you try using Watchdog      Python API library and shell utilities to monitor file system events       Directory monitoring made easy with         A cross-platform API    A shell tool to run commands in response to directory changes          Get started quickly with a simple example in Quickstart

User · Answer

It should not work on windows  maybe with cygwin     but for unix user  you should use the  fcntl  system call  Here is an example in Python  It s mostly the same code if you need to write it in C  same function names   import time import fcntl import os import signal  FNAME     HOME TOTO FILETOWATCH   def handler signum  frame       print  File  s modified     FNAME    signal signal signal SIGIO  handler  fd   os open FNAME   os O RDONLY  fcntl fcntl fd  fcntl F SETSIG  0  fcntl fcntl fd  fcntl F NOTIFY              fcntl DN MODIFY   fcntl DN CREATE   fcntl DN MULTISHOT   while True      time sleep 10000

User · Answer

Simplest solution for me is using watchdog s tool watchmedo  From https   pypi python org pypi watchdog I now have a process that looks up the sql files in a directory and executes them if necessary    watchmedo shell-command   --patterns    sql    --recursive   --command    Desktop load files into mysql database sh

User · Answer

If polling is good enough for you  I d just watch if the  modified time  file stat changes   To read it   os stat filename  st mtime    Also note that the Windows native change event solution does not work in all circumstances  e g  on network drives    import os  class Monkey object       def   init   self           self  cached stamp   0         self filename     path to file       def ook self           stamp   os stat self filename  st mtime         if stamp    self  cached stamp              self  cached stamp   stamp               File has changed  so do something

User · Answer

Check out pyinotify   inotify replaces dnotify  from an earlier answer  in newer linuxes and allows file-level rather than directory-level monitoring

User · Answer

For watching a single file with polling  and minimal dependencies  here is a fully fleshed-out example  based on answer from Deestan  above    import os import sys  import time  class Watcher object       running   True     refresh delay secs   1        Constructor     def   init   self  watch file  call func on change None   args    kwargs           self  cached stamp   0         self filename   watch file         self call func on change   call func on change         self args   args         self kwargs   kwargs        Look for changes     def look self           stamp   os stat self filename  st mtime         if stamp    self  cached stamp              self  cached stamp   stamp               File has changed  so do something                print  File changed               if self call func on change is not None                  self call func on change  self args    self kwargs         Keep watching in a loop             def watch self           while self running               try                     Look for changes                 time sleep self refresh delay secs                   self look                except KeyboardInterrupt                   print   nDone                    break              except FileNotFoundError                    Action on file not found                 pass             except                   print  Unhandled error   s    sys exc info   0      Call this function each time a change happens def custom action text       print text   watch file    my file txt     watcher   Watcher watch file     simple watcher   Watcher watch file  custom action  text  yes  changed      also call custom action function watcher watch      start the watch going

User · Answer

I d try something like this       try              f   open filePath      except IOError              print  No such file   s    filePath             raw input  Press Enter to close window       try              lines   f readlines               while True                      line   f readline                       try                              if not line                                      time sleep 1                              else                                      functionThatAnalisesTheLine line                      except Exception  e                                handle the exception somehow  for example  log the trace  and raise the same exception again                             raw input  Press Enter to close window                               raise e     finally              f close     The loop checks if there is a new line s  since last time file was read - if there is  it s read and passed to the functionThatAnalisesTheLine function  If not  script waits 1 second and retries the process

User · Answer

Well  since you are using Python  you can just open a file and keep reading lines from it   f   open  file log     If the line read is not empty  you process it   line   f readline   if line         Do what you want with the line   You may be missing that it is ok to keep calling readline at the EOF  It will just keep returning an empty string in this case  And when something is appended to the log file  the reading will continue from where it stopped  as you need   If you are looking for a solution that uses events  or a particular library  please specify this in your question  Otherwise  I think this solution is just fine

User · Answer

You can also use a simple library called repyt  here is an example   repyt   app py

User · Answer

Check out pyinotify   inotify replaces dnotify  from an earlier answer  in newer linuxes and allows file-level rather than directory-level monitoring

User · Answer

Check my answer to a similar question  You could try the same loop in Python  This page suggests   import time  while 1      where   file tell       line   file readline       if not line          time sleep 1          file seek where      else          print line    already has newline   Also see the question tail   a file with Python

User · Answer

Did you try using Watchdog      Python API library and shell utilities to monitor file system events       Directory monitoring made easy with         A cross-platform API    A shell tool to run commands in response to directory changes          Get started quickly with a simple example in Quickstart

User · Answer

I don t know any Windows specific function  You could try getting the MD5 hash of the file every second minute hour  depends on how fast you need it  and compare it to the last hash  When it differs you know the file has been changed and you read out the newest lines

User · Answer

related  4Oh4 solution a smooth change for a list of files to watch   import os import sys import time  class Watcher object       running   True     refresh delay secs   1        Constructor     def   init   self  watch files  call func on change None   args    kwargs           self  cached stamp   0         self  cached stamp files              self filenames   watch files         self call func on change   call func on change         self args   args         self kwargs   kwargs        Look for changes     def look self           for file in self filenames              stamp   os stat file  st mtime             if not file in self  cached stamp files                  self  cached stamp files file    0             if stamp    self  cached stamp files file                   self  cached stamp files file    stamp                   File has changed  so do something                    file to read   open file   r                   value   file to read read                   print  value from file   value                  file to read seek 0                  if self call func on change is not None                      self call func on change  self args    self kwargs         Keep watching in a loop     def watch self           while self running              try                    Look for changes                 time sleep self refresh delay secs                  self look               except KeyboardInterrupt                  print   nDone                   break             except FileNotFoundError                    Action on file not found                 pass             except Exception as e                  print e                  print  Unhandled error   s    sys exc info   0      Call this function each time a change happens def custom action text       print text        pass  watch files      Users mexekanez my file txt     Users mexekanez my file1 txt      watcher   Watcher watch file     simple    if   name         main         watcher   Watcher watch files  custom action  text  yes  changed      also call custom action function     watcher watch      start the watch going

User · Answer

You can also use a simple library called repyt  here is an example   repyt   app py

User · Answer

Have you already looked at the documentation available on http   timgolden me uk python win32 how do i watch directory for changes html  If you only need it to work under Windows the 2nd example seems to be exactly what you want  if you exchange the path of the directory with one of the files you want to watch   Otherwise  polling will probably be the only really platform-independent option  Note  I haven t tried any of these solutions

User · Answer

Well after a bit of hacking of Tim Golden s script  I have the following which seems to work quite well   import os  import win32file import win32con  path to watch         look at the current directory file to watch    test txt    look for changes to a file called test txt  def ProcessNewData  newData        print  Text added   s  newData    Set up the bits we ll need for output ACTIONS       1    Created     2    Deleted     3    Updated     4    Renamed from something     5    Renamed to something    FILE LIST DIRECTORY   0x0001 hDir   win32file CreateFile     path to watch    FILE LIST DIRECTORY    win32con FILE SHARE READ   win32con FILE SHARE WRITE    None    win32con OPEN EXISTING    win32con FILE FLAG BACKUP SEMANTICS    None      Open the file we re interested in a   open file to watch   r      Throw away any exising log data a read      Wait for new data and call ProcessNewData for each new chunk that s written while 1      Wait for a change to occur   results   win32file ReadDirectoryChangesW       hDir      1024      False      win32con FILE NOTIFY CHANGE LAST WRITE      None      None          For each change  check to see if it s updating the file we re interested in   for action  file in results      full filename   os path join  path to watch  file       print file  ACTIONS get  action   Unknown       if file    file to watch          newText   a read           if newText                    ProcessNewData  newText     It could probably do with a load more error checking  but for simply watching a log file and doing some processing on it before spitting it out to the screen  this works well   Thanks everyone for your input - great stuff

[python] How do I watch a file for changes?

Examples related to python

Examples related to file

Examples related to pywin32

Examples related to watch