ProcessStartInfo hanging on WaitForExit Why

Question

I have the following code   info   new System Diagnostics ProcessStartInfo  TheProgram exe   String Join      args    info CreateNoWindow   true  info WindowStyle   System Diagnostics ProcessWindowStyle Hidden  info RedirectStandardOutput   true  info UseShellExecute   false  System Diagnostics Process p   System Diagnostics Process Start info   p WaitForExit    Console WriteLine p StandardOutput ReadToEnd       need the StandardOutput contents   I know that the output from the process I am starting is around 7MB long  Running it in the Windows console works fine  Unfortunately programmatically this hangs indefinitely at WaitForExit  Note also this does code NOT hang for smaller outputs  like 3KB    Is it possible that the internal StandardOutput in ProcessStartInfo can t buffer 7MB  If so  what should I do instead  If not  what am I doing wrong

User · Answer

I tried to make a class that would solve your problem using asynchronous stream read, by taking in account Mark Byers, Rob, stevejay answers. Doing so I realised that there is a bug related to asynchronous process output stream read.

I reported that bug at Microsoft: https://connect.microsoft.com/VisualStudio/feedback/details/3119134

Summary:

You can't do that:

process.BeginOutputReadLine(); process.Start();

You will receive System.InvalidOperationException : StandardOut has not been redirected or the process hasn't started yet.

============================================================================================================================

Then you have to start asynchronous output read after the process is started:

process.Start(); process.BeginOutputReadLine();

Doing so, make a race condition because the output stream can receive data before you set it to asynchronous:

process.Start(); 
// Here the operating system could give the cpu to another thread.  
// For example, the newly created thread (Process) and it could start writing to the output
// immediately before next line would execute. 
// That create a race condition.
process.BeginOutputReadLine();

============================================================================================================================

Then some people could say that you just have to read the stream before you set it to asynchronous. But the same problem occurs. There will be a race condition between the synchronous read and set the stream into asynchronous mode.

============================================================================================================================

There is no way to acheive safe asynchronous read of an output stream of a process in the actual way "Process" and "ProcessStartInfo" has been designed.

You are probably better using asynchronous read like suggested by other users for your case. But you should be aware that you could miss some information due to race condition.

User · Answer

The documentation for Process StandardOutput says to read before you wait otherwise you can deadlock  snippet copied below       Start the child process   Process p   new Process        Redirect the output stream of the child process   p StartInfo UseShellExecute   false   p StartInfo RedirectStandardOutput   true   p StartInfo FileName    Write500Lines exe    p Start        Do not wait for the child process to exit before     reading to the end of its redirected stream      p WaitForExit        Read the output stream first and then wait   string output   p StandardOutput ReadToEnd     p WaitForExit

User · Answer

After reading all the posts here  i settled on the consolidated solution of Marko Avlija    However  it did not solve all of my issues   In our environment we have a Windows Service which is scheduled to run hundreds of different  bat  cmd  exe     etc  files which have accumulated over the years and were written by many different people and in different styles  We have no control over the writing of the programs  amp  scripts  we are just responsible for scheduling  running  and reporting on success failure   So i tried pretty much all of the suggestions here with different levels of success  Marko s answer was almost perfect  but when run as a service  it didnt always capture stdout  I never got to the bottom of why not   The only solution we found that works in ALL our cases is this   http   csharptest net 319 using-the-processrunner-class index html

User · Answer

The problem is that if you redirect StandardOutput and or StandardError the internal buffer can become full  Whatever order you use  there can be a problem    If you wait for the process to exit before reading StandardOutput the process can block trying to write to it  so the process never ends  If you read from StandardOutput using ReadToEnd then your process can block if the process never closes StandardOutput  for example if it never terminates  or if it is blocked writing to StandardError     The solution is to use asynchronous reads to ensure that the buffer doesn t get full  To avoid any deadlocks and collect up all output from both StandardOutput and StandardError you can do this   EDIT  See answers below for how avoid an ObjectDisposedException if the timeout occurs   using  Process process   new Process          process StartInfo FileName   filename      process StartInfo Arguments   arguments      process StartInfo UseShellExecute   false      process StartInfo RedirectStandardOutput   true      process StartInfo RedirectStandardError   true       StringBuilder output   new StringBuilder        StringBuilder error   new StringBuilder         using  AutoResetEvent outputWaitHandle   new AutoResetEvent false       using  AutoResetEvent errorWaitHandle   new AutoResetEvent false                 process OutputDataReceived     sender  e    gt                if  e Data    null                                outputWaitHandle Set                              else                               output AppendLine e Data                                    process ErrorDataReceived     sender  e    gt                        if  e Data    null                                errorWaitHandle Set                              else                               error AppendLine e Data                                     process Start             process BeginOutputReadLine            process BeginErrorReadLine             if  process WaitForExit timeout   amp  amp              outputWaitHandle WaitOne timeout   amp  amp              errorWaitHandle WaitOne timeout                            Process completed  Check process ExitCode here                    else                          Timed out

User · Answer

I solved it this way               Process proc   new Process                proc StartInfo FileName   batchFile              proc StartInfo UseShellExecute   false              proc StartInfo CreateNoWindow   true              proc StartInfo RedirectStandardError   true              proc StartInfo RedirectStandardInput   true              proc StartInfo RedirectStandardOutput   true              proc StartInfo WindowStyle   ProcessWindowStyle Hidden                    proc Start                StreamWriter streamWriter   proc StandardInput              StreamReader outputReader   proc StandardOutput              StreamReader errorReader   proc StandardError              while   outputReader EndOfStream                                string text   outputReader ReadLine                                        streamWriter WriteLine text                              while   errorReader EndOfStream                                                   string text   errorReader ReadLine                    streamWriter WriteLine text                              streamWriter Close                proc WaitForExit      I redirected both input  output and error and handled reading from output and error streams  This solution works for SDK 7- 8 1  both for Windows 7 and Windows 8

User · Answer

Credit to EM0 for https   stackoverflow com a 17600012 4151626  The other solutions  including EM0 s  still deadlocked for my application  due to internal timeouts and the use of both StandardOutput and StandardError by the spawned application   Here is what worked for me   Process p   new Process       StartInfo   new ProcessStartInfo           FileName   exe      Arguments   args      UseShellExecute   false      RedirectStandardOutput   true      RedirectStandardError   true        p Start     string cv error   null  Thread et   new Thread      gt    cv error   p StandardError ReadToEnd        et Start     string cv out   null  Thread ot   new Thread      gt    cv out   p StandardOutput ReadToEnd        ot Start     p WaitForExit    ot Join    et Join      Edit  added initialization of StartInfo to code sample

User · Answer

The problem with unhandled ObjectDisposedException happens when the process is timed out  In such case the other parts of the condition   if  process WaitForExit timeout        amp  amp  outputWaitHandle WaitOne timeout        amp  amp  errorWaitHandle WaitOne timeout     are not executed  I resolved this problem in a following way   using  AutoResetEvent outputWaitHandle   new AutoResetEvent false   using  AutoResetEvent errorWaitHandle   new AutoResetEvent false         using  Process process   new Process                     preparing ProcessStartInfo          try                       process OutputDataReceived     sender  e    gt                                        if  e Data    null                                                outputWaitHandle Set                                              else                                               outputBuilder AppendLine e Data                                                        process ErrorDataReceived     sender  e    gt                                        if  e Data    null                                                errorWaitHandle Set                                              else                                               errorBuilder AppendLine e Data                                                         process Start                 process BeginOutputReadLine                process BeginErrorReadLine                 if  process WaitForExit timeout                                 exitCode   process ExitCode                            else                                  timed out                            output   outputBuilder ToString                      finally                       outputWaitHandle WaitOne timeout               errorWaitHandle WaitOne timeout

User · Answer

I thing that this is simple and better approach  we don t need AutoResetEvent    public static string GGSCIShell string Path  string Command        using  Process process   new Process                  process StartInfo WorkingDirectory   Path          process StartInfo FileName   Path      ggsci exe           process StartInfo CreateNoWindow   true          process StartInfo RedirectStandardOutput   true          process StartInfo RedirectStandardInput   true          process StartInfo UseShellExecute   false           StringBuilder output   new StringBuilder            process OutputDataReceived     sender  e    gt                        if  e Data    null                                output AppendLine e Data                                     process Start            process StandardInput WriteLine Command           process BeginOutputReadLine              int timeoutParts   10          int timeoutPart    int TIMEOUT   timeoutParts          do                       Thread Sleep 500    sometimes halv scond is enough to empty output buff  therefore  exit  will be accepted without  timeoutPart  waiting              process StandardInput WriteLine  exit                timeoutParts--                    while   process WaitForExit timeoutPart   amp  amp  timeoutParts  gt  0            if  timeoutParts  lt   0                        output AppendLine  ------ GGSCIShell TIMEOUT      TIMEOUT    ms ------                       string result   output ToString            return result

User · Answer

We have this issue as well  or a variant    Try the following   1  Add a timeout to p WaitForExit nnnn   where nnnn is in milliseconds   2  Put the ReadToEnd call before the WaitForExit call   This is what we ve seen MS recommend

User · Answer

This post maybe outdated but i found out the main cause why it usually hang is due to stack overflow for the redirectStandardoutput or if you have redirectStandarderror   As the output data or the error data is large  it will cause a hang time as it is still processing for indefinite duration   so to resolve this issue   p StartInfo RedirectStandardoutput   False p StartInfo RedirectStandarderror   False

User · Answer

This is a more modern awaitable  Task Parallel Library  TPL  based solution for  NET 4 5 and above   Usage Example  try       var exitCode   await StartProcess           dotnet             --version              C             10000           Console Out           Console Out       Console WriteLine   Process Exited with Exit Code  exitCode        catch  TaskCanceledException        Console WriteLine  Process Timed Out         Implementation  public static async Task lt int gt  StartProcess      string filename      string arguments      string workingDirectory  null      int  timeout   null      TextWriter outputTextWriter   null      TextWriter errorTextWriter   null        using  var process   new Process                 StartInfo   new ProcessStartInfo                         CreateNoWindow   true              Arguments   arguments              FileName   filename              RedirectStandardOutput   outputTextWriter    null              RedirectStandardError   errorTextWriter    null              UseShellExecute   false              WorkingDirectory   workingDirectory                                var cancellationTokenSource   timeout HasValue               new CancellationTokenSource timeout Value                new CancellationTokenSource             process Start             var tasks   new List lt Task gt  3    process WaitForExitAsync cancellationTokenSource Token             if  outputTextWriter    null                        tasks Add ReadAsync                  x   gt                                        process OutputDataReceived    x                      process BeginOutputReadLine                                       x   gt  process OutputDataReceived -  x                  outputTextWriter                  cancellationTokenSource Token                       if  errorTextWriter    null                        tasks Add ReadAsync                  x   gt                                        process ErrorDataReceived    x                      process BeginErrorReadLine                                       x   gt  process ErrorDataReceived -  x                  errorTextWriter                  cancellationTokenSource Token                       await Task WhenAll tasks           return process ExitCode                lt summary gt      Waits asynchronously for the process to exit       lt  summary gt       lt param name  process  gt The process to wait for cancellation  lt  param gt       lt param name  cancellationToken  gt A cancellation token  If invoked  the task will return     immediately as cancelled  lt  param gt       lt returns gt A Task representing waiting for the process to end  lt  returns gt  public static Task WaitForExitAsync      this Process process      CancellationToken cancellationToken   default CancellationToken         process EnableRaisingEvents   true       var taskCompletionSource   new TaskCompletionSource lt object gt          EventHandler handler   null      handler    sender  args    gt                process Exited -  handler          taskCompletionSource TrySetResult null              process Exited    handler       if  cancellationToken    default CancellationToken                 cancellationToken Register                   gt                                process Exited -  handler                  taskCompletionSource TrySetCanceled                               return taskCompletionSource Task          lt summary gt      Reads the data from the specified data recieved event and writes it to the      lt paramref name  textWriter   gt        lt  summary gt       lt param name  addHandler  gt Adds the event handler  lt  param gt       lt param name  removeHandler  gt Removes the event handler  lt  param gt       lt param name  textWriter  gt The text writer  lt  param gt       lt param name  cancellationToken  gt The cancellation token  lt  param gt       lt returns gt A task representing the asynchronous operation  lt  returns gt  public static Task ReadAsync      this Action lt DataReceivedEventHandler gt  addHandler      Action lt DataReceivedEventHandler gt  removeHandler      TextWriter textWriter      CancellationToken cancellationToken   default CancellationToken         var taskCompletionSource   new TaskCompletionSource lt object gt          DataReceivedEventHandler handler   null      handler   new DataReceivedEventHandler           sender  e    gt                        if  e Data    null                                removeHandler handler                   taskCompletionSource TrySetResult null                             else                               textWriter WriteLine e Data                                  addHandler handler        if  cancellationToken    default CancellationToken                 cancellationToken Register                   gt                                removeHandler handler                   taskCompletionSource TrySetCanceled                               return taskCompletionSource Task

User · Answer

I know that this is supper old but  after reading this whole page none of the solutions was working for me  although I didn t try Muhammad Rehan as the code was a little hard to follow  although I guess he was on the right track   When I say it didn t work that s not entirely true  sometimes it would work fine  I guess it is something to do with the length of the output before an EOF mark   Anyway  the solution that worked for me was to use different threads to read the StandardOutput and StandardError and write the messages           StreamWriter sw   null          var queue   new ConcurrentQueue lt string gt              var flushTask   new System Timers Timer 50           flushTask Elapsed     s  e    gt                        while   queue IsEmpty                                string line   null                  if  queue TryDequeue out line                       sw WriteLine line                             sw FlushAsync                       flushTask Start             using  var process   new Process                          try                               process StartInfo FileName                           process StartInfo Arguments                           process StartInfo UseShellExecute   false                  process StartInfo RedirectStandardOutput   true                  process StartInfo RedirectStandardError   true                   process Start                     var outputRead   Task Run      gt                                        while   process StandardOutput EndOfStream                                                queue Enqueue process StandardOutput ReadLine                                                                var errorRead   Task Run      gt                                        while   process StandardError EndOfStream                                                queue Enqueue process StandardError ReadLine                                                                var timeout   new TimeSpan hours  0  minutes  10  seconds  0                    if  Task WaitAll new     outputRead  errorRead    timeout   amp  amp                      process WaitForExit  int timeout TotalMilliseconds                                         if  process ExitCode    0                                                throw new Exception   Failed run    blah blah                                                            else                                       throw new Exception   process timed out after waiting  timeout                                                 catch  Exception e                                throw new Exception   Failed to succesfully run the process        e                                   Hope this helps someone  who thought this could be so hard

User · Answer

I ve read many of the answers and made my own  Not sure this one will fix in any case  but it fixes in my environment  I m just not using WaitForExit and use WaitHandle WaitAll on both output  amp  error end signals  I will be glad  if someone will see possible problems with that  Or if it will help someone  For me it s better because not uses timeouts   private static int DoProcess string workingDir  string fileName  string arguments        int exitCode      using  var process   new Process               StartInfo                         WorkingDirectory   workingDir              WindowStyle   ProcessWindowStyle Hidden              CreateNoWindow   true              UseShellExecute   false              FileName   fileName              Arguments   arguments              RedirectStandardError   true              RedirectStandardOutput   true                    EnableRaisingEvents   true                      using  var outputWaitHandle   new AutoResetEvent false           using  var errorWaitHandle   new AutoResetEvent false                         process OutputDataReceived     sender  args    gt                                   ReSharper disable once AccessToDisposedClosure                 if  args Data    null  Debug Log args Data                   else outputWaitHandle Set                               process ErrorDataReceived     sender  args    gt                                   ReSharper disable once AccessToDisposedClosure                 if  args Data    null  Debug LogError args Data                   else errorWaitHandle Set                                process Start                process BeginOutputReadLine                process BeginErrorReadLine                 WaitHandle WaitAll new WaitHandle     outputWaitHandle  errorWaitHandle                  exitCode   process ExitCode                      return exitCode

User · Answer

Mark Byers  answer is excellent  but I would just add the following    The OutputDataReceived and ErrorDataReceived delegates need to be removed before the outputWaitHandle and errorWaitHandle get disposed  If the process continues to output data after the timeout has been exceeded and then terminates  the outputWaitHandle and errorWaitHandle variables will be accessed after being disposed    FYI I had to add this caveat as an answer as I couldn t comment on his post

User · Answer

I think with async  it is possible to have a more elegant solution and not having deadlocks even when using both standardOutput and standardError   using  Process process   new Process          process StartInfo FileName   filename      process StartInfo Arguments   arguments      process StartInfo UseShellExecute   false      process StartInfo RedirectStandardOutput   true      process StartInfo RedirectStandardError   true       process Start         var tStandardOutput   process StandardOutput ReadToEndAsync        var tStandardError   process StandardError ReadToEndAsync         if  process WaitForExit timeout                 string output   await tStandardOutput          string errors   await tStandardError              Process completed  Check process ExitCode here            else                  Timed out            It is base on Mark Byers answer  If you are not in an async method  you can use string output   tStandardOutput result  instead of await

User · Answer

Introduction  Currently accepted answer doesn t work  throws exception  and there are too many workarounds but no complete code  This is obviously wasting lots of people s time because this is a popular question   Combining Mark Byers  answer and Karol Tyl s answer I wrote full code based on how I want to use the Process Start method   Usage  I have used it to create progress dialog around git commands  This is how I ve used it       private bool Run string fullCommand                Error               int timeout   5000           var result   ProcessNoBS Start              filename    C  Program Files Git cmd git exe               arguments  fullCommand              timeoutInMs  timeout              workingDir    C  test             if  result hasTimedOut                        Error   String Format  Timeout   0  sec    timeout 1000               return false                     if  result ExitCode    0                        Error    String IsNullOrWhiteSpace result stderr                      result stdout   result stderr              return false                     return true          In theory you can also combine stdout and stderr  but I haven t tested that   Code  public struct ProcessResult       public string stdout      public string stderr      public bool hasTimedOut      private int  exitCode       public ProcessResult bool hasTimedOut   true                this hasTimedOut   hasTimedOut          stdout   null          stderr   null          exitCode   null             public int ExitCode               get                        if  hasTimedOut                  throw new InvalidOperationException                       There was no exit code - process has timed out                  return  int exitCode                    set                       exitCode   value                     public class ProcessNoBS       public static ProcessResult Start string filename  string arguments          string workingDir   null  int timeoutInMs   5000          bool combineStdoutAndStderr   false                using  AutoResetEvent outputWaitHandle   new AutoResetEvent false           using  AutoResetEvent errorWaitHandle   new AutoResetEvent false                         using  var process   new Process                                  var info   new ProcessStartInfo                     info CreateNoWindow   true                  info FileName   filename                  info Arguments   arguments                  info UseShellExecute   false                  info RedirectStandardOutput   true                  info RedirectStandardError   true                   if  workingDir    null                      info WorkingDirectory   workingDir                   process StartInfo   info                   StringBuilder stdout   new StringBuilder                    StringBuilder stderr   combineStdoutAndStderr                       stdout   new StringBuilder                     var result   new ProcessResult                     try                                       process OutputDataReceived     sender  e    gt                                                if  e Data    null                              outputWaitHandle Set                            else                             stdout AppendLine e Data                                              process ErrorDataReceived     sender  e    gt                                                if  e Data    null                              errorWaitHandle Set                            else                             stderr AppendLine e Data                                               process Start                         process BeginOutputReadLine                        process BeginErrorReadLine                         if  process WaitForExit timeoutInMs                           result ExitCode   process ExitCode                         else process has timed out                         but that s already default ProcessResult                      result stdout   stdout ToString                        if  combineStdoutAndStderr                          result stderr   null                      else                         result stderr   stderr ToString                         return result                                    finally                                       outputWaitHandle WaitOne timeoutInMs                       errorWaitHandle WaitOne timeoutInMs

User · Answer

Let us call the sample code posted here the redirector and the other program the redirected  If it were me then I would probably write a test redirected program that can be used to duplicate the problem   So I did  For test data I used the ECMA-334 C  Language Specificationv PDF  it is about 5MB  The following is the important part of that   StreamReader stream   null  try   stream   new StreamReader Path     catch  Exception ex        Console Error WriteLine  Input open error      ex Message       return    Console SetIn stream   int datasize   0  try       string record   Console ReadLine        while  record    null                datasize    record Length   2          record   Console ReadLine            Console WriteLine record           catch  Exception ex        Console Error WriteLine   Error   ex Message         return      The datasize value does not match the actual file size but that does not matter  It is not clear if a PDF file always uses both CR and LF at the end of lines but that does not matter for this  You can use any other large text file to test with   Using that the sample redirector code hangs when I write the large amount of data but not when I write a small amount   I tried very much to somehow trace the execution of that code and I could not  I commented out the lines of the redirected program that disabled creation of a console for the redirected program to try to get a separate console window but I could not   Then I found How to start a console app in a new window  the parent   s window  or no window  So apparently we cannot  easily  have a separate console when one console program starts another console program without ShellExecute and since ShellExecute does not support redirection we must share a console  even if we specify no window for the other process   I assume that if the redirected program fills up a buffer somewhere then it must wait for the data to be read and if at that point no data is read by the redirector then it is a deadlock   The solution is to not use ReadToEnd and to read the data while the data is being written but it is not necessary to use asynchronous reads  The solution can be quite simple  The following works for me with the 5 MB PDF   ProcessStartInfo info   new ProcessStartInfo TheProgram   info CreateNoWindow   true  info WindowStyle   System Diagnostics ProcessWindowStyle Hidden  info RedirectStandardOutput   true  info UseShellExecute   false  Process p   Process Start info   string record   p StandardOutput ReadLine    while  record    null        Console WriteLine record       record   p StandardOutput ReadLine      p WaitForExit      Another possibility is to use a GUI program to do the redirection  The preceding code works in a WPF application except with obvious modifications

User · Answer

I was having the same issue  but the reason was different  It would however happen under Windows 8  but not under Windows 7  The following line seems to have caused the problem   pProcess StartInfo UseShellExecute   False   The solution was to NOT disable UseShellExecute  I now received a Shell popup window  which is unwanted  but much better than the program waiting for nothing particular to happen  So I added the following work-around for that   pProcess StartInfo WindowStyle   ProcessWindowStyle Hidden   Now the only thing bothering me is to why this is happening under Windows 8 in the first place

User · Answer

Rob answered it and saved me few more hours of trials  Read the output error buffer before waiting      Read the output stream first and then wait  string output   p StandardOutput ReadToEnd    p WaitForExit

User · Answer

None of the answers above is doing the job   Rob solution hangs and  Mark Byers  solution get the disposed exception  I tried the  solutions  of the other answers    So I decided to suggest another solution   public void GetProcessOutputWithTimeout Process process  int timeoutSec  CancellationToken token  out string output  out int exitCode        string outputLocal        int localExitCode   -1      var task   System Threading Tasks Task Factory StartNew      gt                outputLocal   process StandardOutput ReadToEnd            process WaitForExit            localExitCode   process ExitCode         token        if  task Wait timeoutSec  token                 output   outputLocal          exitCode   localExitCode            else               exitCode   -1          output                using  var process   new Process          process StartInfo            process Start        string outputUnicode  int exitCode      GetProcessOutputWithTimeout process  PROCESS TIMEOUT  out outputUnicode  out exitCode       This code debugged and works perfectly

User · Answer

Workaround I ended up using to avoid all the complexity   var outputFile   Path GetTempFileName    info   new System Diagnostics ProcessStartInfo  TheProgram exe   String Join      args       gt      outputFile     2 gt  amp 1    info CreateNoWindow   true  info WindowStyle   System Diagnostics ProcessWindowStyle Hidden  info UseShellExecute   false  System Diagnostics Process p   System Diagnostics Process Start info   p WaitForExit    Console WriteLine File ReadAllText outputFile      need the StandardOutput contents   So I create a temp file  redirect both the output and error to it by using   gt  outputfile  gt  2 gt  amp 1 and then just read the file after the process has finished   The other solutions are fine for scenarios where you want to do other stuff with the output  but for simple stuff this avoids a lot of complexity

[c#] ProcessStartInfo hanging on "WaitForExit"? Why?

Examples related to c#

Examples related to processstartinfo