Escaping Double Quotes in Batch Script

Question

How would I go about replacing all of the double quotes in my batch file s parameters with escaped double quotes  This is my current batch file  which expands all of its command line parameters inside the string    echo off call bash --verbose -c  g  -linux-4 1       It then uses that string to make a call to Cygwin s bash  executing a Linux cross-compiler  Unfortunately  I m getting parameters like these passed in to my batch file    launch-linux-g   bat  -ftemplate-depth-128 -O3 -finline-functions  -Wno-inline -Wall  -DNDEBUG   -c  -o  C  Users Me Documents Testing SparseLib bin Win32 LinuxRelease hello o    c  Users Me Documents Testing SparseLib SparseLib hello cpp    Where the first quote around the first path passed in is prematurely ending the string being passed to GCC  and passing the rest of the parameters directly to bash  which fails spectacularly    I imagine if I can concatenate the parameters into a single string then escape the quotes it should work fine  but I m having difficulty determining how to do this  Does anyone know

User · Answer

For example for Unreal engine Automation tool run from batch file - this worked for me

eg: -cmdline=" -Messaging" -device=device -addcmdline="-SessionId=session -SessionOwner='owner' -SessionName='Build' -dataProviderMode=local -LogCmds='LogCommodity OFF' -execcmds='automation list; runtests tests+separated+by+T1+T2; quit' " -run

Hope this helps someone, worked for me.

User · Answer

As an addition to mklement0 s excellent answer   Almost all executables accept    as an escaped    Safe usage in cmd however is almost only possible using DELAYEDEXPANSION  To explicitely send a literal   to some process  assign    to an environment variable  and then use that variable  whenever you need to pass a quote  Example   SETLOCAL ENABLEDELAYEDEXPANSION set q    child  malicious argument q  amp whoami    Note SETLOCAL ENABLEDELAYEDEXPANSION seems to work only within batch files  To get DELAYEDEXPANSION in an interactive session  start cmd  V ON   If your batchfile does t work with DELAYEDEXPANSION  you can enable it temporarily     region without DELAYEDEXPANSION  SETLOCAL ENABLEDELAYEDEXPANSION   region with DELAYEDEXPANSION set q    echoarg exe  ab  q   amp  echo danger  ENDLOCAL    region without DELAYEDEXPANSION   If you want to pass dynamic content from a variable that contains quotes that are escaped as    you can replace    with    on expansion   SETLOCAL ENABLEDELAYEDEXPANSION foo exe  danger  amp  bar region with  dynamic content         amp  danger  ENDLOCAL   This replacement is not safe with       style expansion   In case of OP bash -c  g  -linux-4 1  v params        is the safe version     If for some reason even temporarily enabling DELAYEDEXPANSION is not an option  read on   Using    from within cmd is a little bit safer if one always needs to escape special characters  instead of just sometimes   It s less likely to forget a caret  if it s consistent      To achieve this  one precedes any quote with a caret       quotes that should reach the child process as literals must additionally be escaped with a backlash        ALL shell meta characters must be escaped with   as well  e g   amp       amp             gt       gt   etc   Example   child   malicious argument     amp whoami     Source  Everyone quotes command line arguments the wrong way  see  A better method of quoting     To pass dynamic content  one needs to ensure the following  The part of the command that contains the variable must be considered  quoted  by cmd exe  This is impossible if the variable can contain quotes - don t write  var          To achieve this  the last   before the variable and the first   after the variable are not  -escaped  cmd-metacharacters between those two   must not be escaped  Example    foo exe   danger   amp  bar   region with  dynamic content   amp  danger       This isn t safe  if  dynamic content  can contain unmatched quotes

User · Answer

If the string is already within quotes then use another quote to nullify its action  echo  quot Insert tablename col1  Values   quot  quot val1 quot  quot    quot

User · Answer

eplawless s own answer simply and effectively solves his specific problem  it replaces all   instances in the entire argument list with     which is how Bash requires double-quotes inside a double-quoted string to be represented   To generally answer the question of how to escape double-quotes inside a double-quoted string using cmd exe  the Windows command-line interpreter  whether on the command line - often still mistakenly called the  DOS prompt  - or in a batch file  See bottom for a look at PowerShell   tl dr    You must use    when passing a string to a nother  batch file and you may use    with applications created with Microsoft s C C    NET compilers  which also accept      which on Windows includes Python and Node js    Example  foo bat  We had 3   of rain   The following applies to batch files only       is the only way to get the command interpreter  cmd exe  to treat the whole double-quoted string as a single argument  Sadly  however  not only are the enclosing double-quotes retained  as usual   but so are the doubled escaped ones  so obtaining the intended string is a two-step process  e g   assuming that the double-quoted string is passed as the 1st argument   1  set  str   1  removes the enclosing double-quotes  set  str  str        then converts the doubled double-quotes to single ones  Be sure to use the enclosing double-quotes around the assignment parts to prevent unwanted interpretation of the values       is required - as the only option - by many other programs   e g   Ruby  Perl  and even Microsoft s own Windows PowerShell      but ITS USE IS NOT SAFE       is what many executables and interpreters either require - including Windows PowerShell - when passed strings from the outside - or  in the case of Microsoft s compilers  support as an alternative to    - ultimately  though  it s up to the target program to parse the argument list    Example  foo exe  We had 3   of rain    HOWEVER  USE OF    CAN RESULT IN UNWANTED  ARBITRARY EXECUTION OF COMMANDS and or INPUT OUTPUT REDIRECTIONS    The following characters present this risk   amp     lt   gt  For instance  the following results in unintended execution of the ver command  see further below for an explanation and the next bullet point for a workaround    foo exe  3   of snow    amp  ver     For Windows PowerShell      and      are robust  but limited alternatives  see section  Calling PowerShell s CLI      below    If you must use     there are only 3 safe approaches  which are  however quite cumbersome  Tip of the hat to T S for his help    Using  possibly selective  delayed variable expansion in your batch file  you can store literal    in a variable and reference that variable inside a       string using  var  syntax - see T S s helpful answer    The above approach  despite being cumbersome  has the advantage that you can apply it methodically and that it works robustly  with any input   Only with LITERAL strings - ones NOT involving VARIABLES - do you get a similarly methodical approach  categorically  -escape all cmd exe metacharacters     amp     lt   gt  and - if you also want to suppress variable expansion -    foo exe   3    of snow       amp  ver    Otherwise  you must formulate your string based on recognizing which portions of the string cmd exe considers unquoted due to misinterpreting    as closing delimiters    in literal portions containing shell metacharacters    -escape  them  using the example above  it is  amp  that must be  -escaped  foo exe  3   of snow     amp  ver   in portions with      -style variable references  ensure that cmd exe considers them part of a       string and that that the variable values do not themselves have embedded  unbalanced quotes - which is not even always possible      For background information  read on     Background  Note  This is based on my own experiments  Do let me know if I m wrong   POSIX-like shells such as Bash on Unix-like systems tokenize the argument list  string  before passing arguments individually to the target program  among other expansions  they split the argument list into individual words  word splitting  and remove quoting characters from the resulting words  quote removal   The target program is handed an array of individual arguments  with syntactic quotes removed   By contrast  the Windows command interpreter apparently does not tokenize the argument list and simply passes the single string comprising all arguments - including quoting chars  - to the target program  However  some preprocessing takes place before the single string is passed to the target program    escape chars  outside of double-quoted strings are removed  they escape the following char    and variable references  e g    USERNAME   are interpolated first   Thus  unlike in Unix  it is the target program s responsibility to parse to parse the arguments string and break it down into individual arguments with quotes removed  Thus  different programs can hypothetically require differing escaping methods and there s no single escaping mechanism that is guaranteed to work with all programs - https   stackoverflow com a 4094897 45375 contains excellent background on the anarchy that is Windows command-line parsing     In practice     is very common  but NOT SAFE  as mentioned above   Since cmd exe itself doesn t recognize    as an escaped double-quote  it can misconstrue later tokens on the command line as unquoted and potentially interpret them as commands and or input output redirections  In a nutshell  the problem surfaces  if any of the following characters follow an opening or unbalanced      amp     lt   gt   for example   foo exe  3   of snow    amp  ver     cmd exe sees the following tokens  resulting from misinterpreting    as a regular double-quote     3   of snow    rest   amp  ver    Since cmd exe thinks that  amp  ver  is unquoted  it interprets it as  amp   the command-sequencing operator   followed by the name of a command to execute  ver  - the   is ignored  ver reports cmd exe s version information   The overall effect is    First  foo exe is invoked with the first 3 tokens only  Then  command ver is executed    Even in cases where the accidental command does no harm  your overall command won t work as designed  given that not all arguments are passed to it   Many compilers   interpreters recognize ONLY    - e g   the GNU C C   compiler  Python  Perl  Ruby  even Microsoft s own Windows PowerShell when invoked from cmd exe - and  except  with limitations  for Windows PowerShell with      for them there is no simple solution to this problem  Essentially  you d have to know in advance which portions of your command line are misinterpreted as unquoted  and selectively  -escape all instances of  amp     lt   gt  in those portions   By contrast  use of    is SAFE  but is regrettably only supported by Microsoft-compiler-based executables and batch files  in the case of batch files  with the quirks discussed above   which notable excludes PowerShell - see next section     Calling PowerShell s CLI from cmd exe or POSIX-like shells   Note  See the bottom section for how quoting is handled inside PowerShell   When invoked from the outside - e g   from cmd exe  whether from the command line or a batch file    PowerShell  Core  v6  now properly recognizes     in addition to      which is both safe to use and whitespace-preserving    pwsh -c     a  amp   c   length   doesn t break and correctly yields 6  Windows PowerShell  the legacy edition whose last version is 5 1   recognizes only    and  on Windows also     and the more robust             even though internally PowerShell uses   as the escape character in double-quoted strings and also accepts    - see bottom section       Calling Windows PowerShell from cmd exe   a batch file       breaks  because it is fundamentally unsupported    powershell -c     ab  c   length   -  error  The string is missing the terminator      and     work in principle  but aren t safe    powershell -c     ab  c   length   works as intended  it outputs 5  note the 2 spaces  But it isn t safe  because cmd exe metacharacters break the command  unless escaped  powershell -c     a amp   c   length   breaks  due to the  amp   which would have to be escaped as   amp       is safe  but normalize interior whitespace  which can be undesired    powershell -c      a amp   c    length   outputs 4     because the 2 spaces are normalized to 1        is the best choice for Windows PowerShell specifically  where it is both safe and whitespace-preserving  but with PowerShell Core  on Windows  it is the same as      i e  whitespace-normalizing  Credit goes to Venryx for discovering this approach    powershell -c       a amp   c     length   works  doesn t break - despite  amp  - and outputs 5  i e   correctly preserved whitespace  PowerShell Core  pwsh -c       a amp   c     length   works  but outputs 4  i e  normalizes whitespace  as     does     On Unix-like platforms  Linux  macOS   when calling PowerShell  Core  s CLI  pwsh  from a POSIX-like shell such as bash   You must use     which  however is both safe and whitespace-preserving     pwsh -c     a amp   c    length    OK  5     Related information     can only be used as the escape character in unquoted strings - inside double-quoted strings    is not special and treated as a literal    CAVEAT  Use of   in parameters passed to the call statement is broken  this applies to both uses of call  invoking another batch file or binary  and calling a subroutine in the same batch file       instances in double-quoted values are inexplicably doubled  altering the value being passed  e g   if variable  v  contains literal value a b  call  foo   v   assigns  a  b     to  1  the first parameter  in subroutine  foo  Unquoted use of   with call is broken altogether in that   can no longer be used to escape special characters  e g   call foo cmd a  amp b quietly breaks  instead of passing literal a amp b too foo cmd  as would be the case without call  - foo cmd is never even invoked     at least on Windows 7    Escaping a literal   is a special case  unfortunately  which requires distinct syntax depending on whether a string is specified on the command line vs  inside a batch file  see https   stackoverflow com a 31420292 45375   The short of it  Inside a batch file  use     On the command line    cannot be escaped  but if you place a   at the start  end  or inside a variable name in an unquoted string  e g   echo   foo    you can prevent variable expansion  interpolation     instances on the command line that are not part of a variable reference are treated as literals  e g  100     Generally  to safely work with variable values that may contain spaces and special characters    Assignment  Enclose both the variable name and the value in a single pair of double-quotes  e g   set  v a  amp  b  assigns literal value a  amp  b to variable  v   by contrast  set v  a  amp  b  would make the double-quotes part of the value   Escape literal   instances as     works only in batch files - see above   Reference  Double-quote variable references to make sure their value is not interpolated  e g   echo   v   does not subject the value of  v  to interpolation and prints  a  amp  b   but note that the double-quotes are invariably printed too   By contrast  echo  v  passes literal a to echo  interprets  amp  as the command-sequencing operator  and therefore tries to execute a command named b  Also note the above caveat re use of   with the call statement  External programs typically take care of removing enclosing double-quotes around parameters  but  as noted  in batch files you have to do it yourself  e g     1 to remove enclosing double-quotes from the 1st parameter  and  sadly  there is no direct way that I know of to get echo to print a variable value faithfully without the enclosing double-quotes    Neil offers a for-based workaround that works as long as the value has no embedded double quotes  e g   set  var   amp          for  f  delims     v in    var    do echo    v   cmd exe does not recognize single-quotes as string delimiters - they are treated as literals and cannot generally be used to delimit strings with embedded whitespace  also  it follows that the tokens abutting the single-quotes and any tokens in between are treated as unquoted by cmd exe and interpreted accordingly    However  given that target programs ultimately perform their own argument parsing  some programs such as Ruby do recognize single-quoted strings even on Windows  by contrast  C C   executables  Perl and Python do not recognize them  Even if supported by the target program  however  it is not advisable to use single-quoted strings  given that their contents are not protected from potentially unwanted interpretation by cmd exe       Quoting from within PowerShell   Windows PowerShell is a much more advanced shell than cmd exe  and it has been a part of Windows for many years now  and PowerShell Core brought the PowerShell experience to macOS and Linux as well    PowerShell works consistently internally with respect to quoting    inside double-quoted strings  use    or    to escape double-quotes inside single-quoted strings  use    to escape single-quotes   This works on the PowerShell command line and when passing parameters to PowerShell scripts or functions from within PowerShell    As discussed above  passing an escaped double-quote to PowerShell from the outside requires    or  more robustly      - nothing else works    Sadly  when invoking external programs from PowerShell  you re faced with the need to both accommodate PowerShell s own quoting rules and to escape for the target program   This problematic behavior is also discussed and summarized in this answer  Double-quotes inside double-quoted strings   Consider string  3   of rain   which PowerShell-internally translates to literal 3  of rain   If you want to pass this string to an external program  you have to apply the target program s escaping in addition to PowerShell s  say you want to pass the string to a C program  which expects embedded double-quotes to be escaped as      foo exe  3    of rain    Note how both    - to make PowerShell happy - and the   - to make the target program happy - must be present     The same logic applies to invoking a batch file  where    must be used   foo bat  3     of rain    By contrast  embedding single-quotes in a double-quoted string requires no escaping at all   Single-quotes inside single-quoted strings do not require extra escaping  consider  2   of snow   which is PowerShell  representation of 2  of snow   foo exe  2   of snow  foo bat  2   of snow    PowerShell translates single-quoted strings to double-quoted ones before passing them to the target program   However  double-quotes inside single-quoted strings  which do not need escaping for PowerShell  do still need to be escaped for the target program   foo exe  3   of rain  foo bat  3   of rain    PowerShell v3 introduced the magic --  option  called the stop-parsing symbol  which alleviates some of the pain  by passing anything after it uninterpreted to the target program  save for cmd exe-style environment-variable references  e g    USERNAME    which are expanded  e g    foo exe --   3   of rain  -u  USERNAME    Note how escaping the embedded   as    for the target program only  and not also for PowerShell as      is sufficient   However  this approach    does not allow for escaping   characters in order to avoid environment-variable expansions  precludes direct use of PowerShell variables and expressions  instead  the command line must be built in a string variable in a first step  and then invoked with Invoke-Expression in a second      Thus  despite its many advancements  PowerShell has not made escaping much easier when calling external programs  It has  however  introduced support for single-quoted strings   I wonder if it s fundamentally possible in the Windows world to ever switch to the Unix model of letting the shell do all the tokenization and quote removal predictably  up front  irrespective of the target program  and then invoke the target program by passing the resulting tokens

User · Answer

The escape character in batch scripts is     But for double-quoted strings  double up the quotes    string with an embedded    character

User · Answer

Google eventually came up with the answer  The syntax for string replacement in batch is this   set v myvar replace me set v myvar  v myvar ace icate    Which produces  replicate me   My script now looks like this    echo off set v params    set v params  v params       call bash -c  g  -linux-4 1  v params     Which replaces all instances of   with     properly escaped for bash

[scripting] Escaping Double Quotes in Batch Script

Examples related to scripting

Examples related to parameters

Examples related to batch-file

Examples related to escaping

Examples related to quotes