[r] R: invalid multibyte string

I use read.delim(filename) without any parameters to read a tab delimited text file in R.

df = read.delim(file)

This worked as intended. Now I have a weird error message and I can't make any sense of it:

Error in type.convert(data[[i]], as.is = as.is[i], dec = dec, na.strings = character(0L)) : 
invalid multibyte string at '<fd>'
Calls: read.delim -> read.table -> type.convert
Execution halted

Can anybody explain what a multibyte string is? What does fd mean? Are there other ways to read a tab file in R? I have column headers and lines which do not have data for all columns.

I figured out Leafpad to be an adequate and simple text-editor to view and save/convert in certain character sets - at least in the linux-world.

I used this to save the Latin-15 to UTF-8 and it worked.

This happened to me because I had the 'copyright' symbol in one of my strings! Once it was removed, problem solved.

A good rule of thumb, make sure that characters not appearing on your keyboard are removed if you are seeing this error.

I had a similarly strange problem with a file from the program e-prime (edat -> SPSS conversion), but then I discovered that there are many additional encodings you can use. this did the trick for me:

tbl <- read.delim("dir/file.txt", fileEncoding="UCS-2LE")

If you want an R solution, here's a small convenience function I sometimes use to find where the offending (multiByte) character is lurking. Note that it is the next character to what gets printed. This works because print will work fine, but substr throws an error when multibyte characters are present.

find_offending_character <- function(x, maxStringLength=256){  
  print(x)
  for (c in 1:maxStringLength){
    offendingChar <- substr(x,c,c)
    #print(offendingChar) #uncomment if you want the indiv characters printed
    #the next character is the offending multibyte Character
  }    
}

string_vector <- c("test", "Se\x96ora", "works fine")

lapply(string_vector, find_offending_character)

I fix that character and run this again. Hope that helps someone who encounters the invalid multibyte string error.