[vim] How to replace a character by a newline in Vim

I'm trying to replace each , in the current file by a new line:

:%s/,/\n/g 

But it inserts what looks like a ^@ instead of an actual newline. The file is not in DOS mode or anything.

What should I do?

If you are curious, like me, check the question Why is \r a newline for Vim? as well.

From Eclipse, the ^M characters can be embedded in a line, and you want to convert them to newlines.

:s/\r/\r/g

\r can do the work here for you.

You need to use:

:%s/,/^M/g

To get the ^M character, press Ctrl + v followed by Enter.

In the syntax s/foo/bar, \r and \n have different meanings, depending on context.

Short:

For foo:

\r == "carriage return" (CR / ^M)
\n == matches "line feed" (LF) on Linux/Mac, and CRLF on Windows

For bar:

\r == produces LF on Linux/Mac, CRLF on Windows
\n == "null byte" (NUL / ^@)

When editing files in linux (i.e. on a webserver) that were initially created in a windows environment and uploaded (i.e. FTP/SFTP) - all the ^M's you see in vim, are the CR's which linux does not translate as it uses only LF's to depict a line break.

Longer (with ASCII numbers):

NUL == 0x00 == 0 == Ctrl + @ == ^@ shown in vim
LF == 0x0A == 10 == Ctrl + J
CR == 0x0D == 13 == Ctrl + M == ^M shown in vim

Here is a list of the ASCII control characters. Insert them in Vim via Ctrl + V,Ctrl + ---key---.

In Bash or the other Unix/Linux shells, just type Ctrl + ---key---.

Try Ctrl + M in Bash. It's the same as hitting Enter, as the shell realizes what is meant, even though Linux systems use line feeds for line delimiting.

To insert literal's in bash, prepending them with Ctrl + V will also work.

Try in Bash:

echo ^[[33;1mcolored.^[[0mnot colored.

This uses ANSI escape sequences. Insert the two ^['s via Ctrl + V, Esc.

You might also try Ctrl + V,Ctrl + M, Enter, which will give you this:

bash: $'\r': command not found

Remember the \r from above? :>

This ASCII control characters list is different from a complete ASCII symbol table, in that the control characters, which are inserted into a console/pseudoterminal/Vim via the Ctrl key (haha), can be found there.

Whereas in C and most other languages, you usually use the octal codes to represent these 'characters'.

If you really want to know where all this comes from: The TTY demystified. This is the best link you will come across about this topic, but beware: There be dragons.

TL;DR

Usually foo = \n, and bar = \r.

You need to use:

:%s/,/^M/g

To get the ^M character, press Ctrl + v followed by Enter.

With Vim on Windows, use Ctrl + Q in place of Ctrl + V.

But if one has to substitute, then the following thing works:

:%s/\n/\r\|\-\r/g

In the above, every next line is substituted with next line, and then |- and again a new line. This is used in wiki tables.

If the text is as follows:

line1
line2
line3

It is changed to

line1
|-
line2
|-
line3

You need to use:

:%s/,/^M/g

To get the ^M character, press Ctrl + v followed by Enter.

Here's the answer that worked for me. From this guy:

----quoting Use the vi editor to insert a newline char in replace

Something else I have to do and cannot remember and then have to look up.

In vi, to insert a newline character in a search and replace, do the following:

:%s/look_for/replace_with^M/g

The command above would replace all instances of “look_for” with “replace_with\n” (with \n meaning newline).

To get the “^M”, enter the key combination Ctrl + V, and then after that (release all keys) press the Enter key.

Here's the trick:

First, set your Vi(m) session to allow pattern matching with special characters (i.e.: newline). It's probably worth putting this line in your .vimrc or .exrc file:

:set magic

Next, do:

:s/,/,^M/g

To get the ^M character, type Ctrl + V and hit Enter. Under Windows, do Ctrl + Q, Enter. The only way I can remember these is by remembering how little sense they make:

A: What would be the worst control-character to use to represent a newline?

B: Either q (because it usually means "Quit") or v because it would be so easy to type Ctrl + C by mistake and kill the editor.

A: Make it so.

You need to use:

:%s/,/^M/g

To get the ^M character, press Ctrl + v followed by Enter.

If you need to do it for a whole file, it was also suggested to me that you could try from the command line:

sed 's/\\n/\n/g' file > newfile

This is the best answer for the way I think, but it would have been nicer in a table:

Why is \r a newline for Vim?

So, rewording:

You need to use \r to use a line feed (ASCII 0x0A, the Unix newline) in a regex replacement, but that is peculiar to the replacement - you should normally continue to expect to use \n for line feed and \r for carriage return.

This is because Vim used \n in a replacement to mean the NIL character (ASCII 0x00). You might have expected NIL to have been \0 instead, freeing \n for its usual use for line feed, but \0 already has a meaning in regex replacements, so it was shifted to \n. Hence then going further to also shift the newline from \n to \r (which in a regex pattern is the carriage return character, ASCII 0x0D).

Character                | ASCII code | C representation | Regex match | Regex replacement
-------------------------+------------+------------------+-------------+------------------------
nil                      | 0x00       | \0               | \0          | \n
line feed (Unix newline) | 0x0a       | \n               | \n          | \r
carriage return          | 0x0d       | \r               | \r          | <unknown>

NB: ^M (Ctrl + V Ctrl + M on Linux) inserts a newline when used in a regex replacement rather than a carriage return as others have advised (I just tried it).

Also note that Vim will translate the line feed character when it saves to file based on its file format settings and that might confuse matters.

\r can do the work here for you.

With Vim on Windows, use Ctrl + Q in place of Ctrl + V.

This is the best answer for the way I think, but it would have been nicer in a table:

Why is \r a newline for Vim?

So, rewording:

You need to use \r to use a line feed (ASCII 0x0A, the Unix newline) in a regex replacement, but that is peculiar to the replacement - you should normally continue to expect to use \n for line feed and \r for carriage return.

This is because Vim used \n in a replacement to mean the NIL character (ASCII 0x00). You might have expected NIL to have been \0 instead, freeing \n for its usual use for line feed, but \0 already has a meaning in regex replacements, so it was shifted to \n. Hence then going further to also shift the newline from \n to \r (which in a regex pattern is the carriage return character, ASCII 0x0D).

Character                | ASCII code | C representation | Regex match | Regex replacement
-------------------------+------------+------------------+-------------+------------------------
nil                      | 0x00       | \0               | \0          | \n
line feed (Unix newline) | 0x0a       | \n               | \n          | \r
carriage return          | 0x0d       | \r               | \r          | <unknown>

NB: ^M (Ctrl + V Ctrl + M on Linux) inserts a newline when used in a regex replacement rather than a carriage return as others have advised (I just tried it).

Also note that Vim will translate the line feed character when it saves to file based on its file format settings and that might confuse matters.

If you need to do it for a whole file, it was also suggested to me that you could try from the command line:

sed 's/\\n/\n/g' file > newfile

\r can do the work here for you.

From Eclipse, the ^M characters can be embedded in a line, and you want to convert them to newlines.

:s/\r/\r/g

But if one has to substitute, then the following thing works:

:%s/\n/\r\|\-\r/g

In the above, every next line is substituted with next line, and then |- and again a new line. This is used in wiki tables.

If the text is as follows:

line1
line2
line3

It is changed to

line1
|-
line2
|-
line3

With Vim on Windows, use Ctrl + Q in place of Ctrl + V.

Here's the answer that worked for me. From this guy:

----quoting Use the vi editor to insert a newline char in replace

Something else I have to do and cannot remember and then have to look up.

In vi, to insert a newline character in a search and replace, do the following:

:%s/look_for/replace_with^M/g

The command above would replace all instances of “look_for” with “replace_with\n” (with \n meaning newline).

To get the “^M”, enter the key combination Ctrl + V, and then after that (release all keys) press the Enter key.

With Vim on Windows, use Ctrl + Q in place of Ctrl + V.

In the syntax s/foo/bar, \r and \n have different meanings, depending on context.

Short:

For foo:

\r == "carriage return" (CR / ^M)
\n == matches "line feed" (LF) on Linux/Mac, and CRLF on Windows

For bar:

\r == produces LF on Linux/Mac, CRLF on Windows
\n == "null byte" (NUL / ^@)

When editing files in linux (i.e. on a webserver) that were initially created in a windows environment and uploaded (i.e. FTP/SFTP) - all the ^M's you see in vim, are the CR's which linux does not translate as it uses only LF's to depict a line break.

Longer (with ASCII numbers):

NUL == 0x00 == 0 == Ctrl + @ == ^@ shown in vim
LF == 0x0A == 10 == Ctrl + J
CR == 0x0D == 13 == Ctrl + M == ^M shown in vim

Here is a list of the ASCII control characters. Insert them in Vim via Ctrl + V,Ctrl + ---key---.

In Bash or the other Unix/Linux shells, just type Ctrl + ---key---.

Try Ctrl + M in Bash. It's the same as hitting Enter, as the shell realizes what is meant, even though Linux systems use line feeds for line delimiting.

To insert literal's in bash, prepending them with Ctrl + V will also work.

Try in Bash:

echo ^[[33;1mcolored.^[[0mnot colored.

This uses ANSI escape sequences. Insert the two ^['s via Ctrl + V, Esc.

You might also try Ctrl + V,Ctrl + M, Enter, which will give you this:

bash: $'\r': command not found

Remember the \r from above? :>

This ASCII control characters list is different from a complete ASCII symbol table, in that the control characters, which are inserted into a console/pseudoterminal/Vim via the Ctrl key (haha), can be found there.

Whereas in C and most other languages, you usually use the octal codes to represent these 'characters'.

If you really want to know where all this comes from: The TTY demystified. This is the best link you will come across about this topic, but beware: There be dragons.

TL;DR

Usually foo = \n, and bar = \r.