[java] How to convert Strings to and from UTF8 byte arrays in Java

In Java, I have a String and I want to encode it as a byte array (in UTF8, or some other encoding). Alternately, I have a byte array (in some known encoding) and I want to convert it into a Java String. How do I do these conversions?

This question is related to java string encoding character-encoding

The answer is


Charset UTF8_CHARSET = Charset.forName("UTF-8");
String strISO = "{\"name\":\"?\"}";
System.out.println(strISO);
byte[] b = strISO.getBytes();
for (byte c: b) {
    System.out.print("[" + c + "]");
}
String str = new String(b, UTF8_CHARSET);
System.out.println(str);

You can convert directly via the String(byte[], String) constructor and getBytes(String) method. Java exposes available character sets via the Charset class. The JDK documentation lists supported encodings.

90% of the time, such conversions are performed on streams, so you'd use the Reader/Writer classes. You would not incrementally decode using the String methods on arbitrary byte streams - you would leave yourself open to bugs involving multibyte characters.


Here's a solution that avoids performing the Charset lookup for every conversion:

import java.nio.charset.Charset;

private final Charset UTF8_CHARSET = Charset.forName("UTF-8");

String decodeUTF8(byte[] bytes) {
    return new String(bytes, UTF8_CHARSET);
}

byte[] encodeUTF8(String string) {
    return string.getBytes(UTF8_CHARSET);
}

If you are using 7-bit ASCII or ISO-8859-1 (an amazingly common format) then you don't have to create a new java.lang.String at all. It's much much more performant to simply cast the byte into char:

Full working example:

for (byte b : new byte[] { 43, 45, (byte) 215, (byte) 247 }) {
    char c = (char) b;
    System.out.print(c);
}

If you are not using extended-characters like Ä, Æ, Å, Ç, Ï, Ê and can be sure that the only transmitted values are of the first 128 Unicode characters, then this code will also work for UTF-8 and extended ASCII (like cp-1252).


String original = "hello world";
byte[] utf8Bytes = original.getBytes("UTF-8");

For decoding a series of bytes to a normal string message I finally got it working with UTF-8 encoding with this code:

/* Convert a list of UTF-8 numbers to a normal String
 * Usefull for decoding a jms message that is delivered as a sequence of bytes instead of plain text
 */
public String convertUtf8NumbersToString(String[] numbers){
    int length = numbers.length;
    byte[] data = new byte[length];

    for(int i = 0; i< length; i++){
        data[i] = Byte.parseByte(numbers[i]);
    }
    return new String(data, Charset.forName("UTF-8"));
}

Reader reader = new BufferedReader(
    new InputStreamReader(
        new ByteArrayInputStream(
            string.getBytes(StandardCharsets.UTF_8)), StandardCharsets.UTF_8));

You can convert directly via the String(byte[], String) constructor and getBytes(String) method. Java exposes available character sets via the Charset class. The JDK documentation lists supported encodings.

90% of the time, such conversions are performed on streams, so you'd use the Reader/Writer classes. You would not incrementally decode using the String methods on arbitrary byte streams - you would leave yourself open to bugs involving multibyte characters.


String original = "hello world";
byte[] utf8Bytes = original.getBytes("UTF-8");

Here's a solution that avoids performing the Charset lookup for every conversion:

import java.nio.charset.Charset;

private final Charset UTF8_CHARSET = Charset.forName("UTF-8");

String decodeUTF8(byte[] bytes) {
    return new String(bytes, UTF8_CHARSET);
}

byte[] encodeUTF8(String string) {
    return string.getBytes(UTF8_CHARSET);
}

terribly late but i just encountered this issue and this is my fix:

private static String removeNonUtf8CompliantCharacters( final String inString ) {
    if (null == inString ) return null;
    byte[] byteArr = inString.getBytes();
    for ( int i=0; i < byteArr.length; i++ ) {
        byte ch= byteArr[i]; 
        // remove any characters outside the valid UTF-8 range as well as all control characters
        // except tabs and new lines
        if ( !( (ch > 31 && ch < 253 ) || ch == '\t' || ch == '\n' || ch == '\r') ) {
            byteArr[i]=' ';
        }
    }
    return new String( byteArr );
}

String original = "hello world";
byte[] utf8Bytes = original.getBytes("UTF-8");

As an alternative, StringUtils from Apache Commons can be used.

 byte[] bytes = {(byte) 1};
 String convertedString = StringUtils.newStringUtf8(bytes);

or

 String myString = "example";
 byte[] convertedBytes = StringUtils.getBytesUtf8(myString);

If you have non-standard charset, you can use getBytesUnchecked() or newString() accordingly.


//query is your json   

 DefaultHttpClient httpClient = new DefaultHttpClient();
 HttpPost postRequest = new HttpPost("http://my.site/test/v1/product/search?qy=");

 StringEntity input = new StringEntity(query, "UTF-8");
 input.setContentType("application/json");
 postRequest.setEntity(input);   
 HttpResponse response=response = httpClient.execute(postRequest);

You can convert directly via the String(byte[], String) constructor and getBytes(String) method. Java exposes available character sets via the Charset class. The JDK documentation lists supported encodings.

90% of the time, such conversions are performed on streams, so you'd use the Reader/Writer classes. You would not incrementally decode using the String methods on arbitrary byte streams - you would leave yourself open to bugs involving multibyte characters.


My tomcat7 implementation is accepting strings as ISO-8859-1; despite the content-type of the HTTP request. The following solution worked for me when trying to correctly interpret characters like 'é' .

byte[] b1 = szP1.getBytes("ISO-8859-1");
System.out.println(b1.toString());

String szUT8 = new String(b1, "UTF-8");
System.out.println(szUT8);

When trying to interpret the string as US-ASCII, the byte info wasn't correctly interpreted.

b1 = szP1.getBytes("US-ASCII");
System.out.println(b1.toString());

Reader reader = new BufferedReader(
    new InputStreamReader(
        new ByteArrayInputStream(
            string.getBytes(StandardCharsets.UTF_8)), StandardCharsets.UTF_8));

You can convert directly via the String(byte[], String) constructor and getBytes(String) method. Java exposes available character sets via the Charset class. The JDK documentation lists supported encodings.

90% of the time, such conversions are performed on streams, so you'd use the Reader/Writer classes. You would not incrementally decode using the String methods on arbitrary byte streams - you would leave yourself open to bugs involving multibyte characters.


If you are using 7-bit ASCII or ISO-8859-1 (an amazingly common format) then you don't have to create a new java.lang.String at all. It's much much more performant to simply cast the byte into char:

Full working example:

for (byte b : new byte[] { 43, 45, (byte) 215, (byte) 247 }) {
    char c = (char) b;
    System.out.print(c);
}

If you are not using extended-characters like Ä, Æ, Å, Ç, Ï, Ê and can be sure that the only transmitted values are of the first 128 Unicode characters, then this code will also work for UTF-8 and extended ASCII (like cp-1252).


terribly late but i just encountered this issue and this is my fix:

private static String removeNonUtf8CompliantCharacters( final String inString ) {
    if (null == inString ) return null;
    byte[] byteArr = inString.getBytes();
    for ( int i=0; i < byteArr.length; i++ ) {
        byte ch= byteArr[i]; 
        // remove any characters outside the valid UTF-8 range as well as all control characters
        // except tabs and new lines
        if ( !( (ch > 31 && ch < 253 ) || ch == '\t' || ch == '\n' || ch == '\r') ) {
            byteArr[i]=' ';
        }
    }
    return new String( byteArr );
}

I can't comment but don't want to start a new thread. But this isn't working. A simple round trip:

byte[] b = new byte[]{ 0, 0, 0, -127 };  // 0x00000081
String s = new String(b,StandardCharsets.UTF_8); // UTF8 = 0x0000, 0x0000,  0x0000, 0xfffd
b = s.getBytes(StandardCharsets.UTF_8); // [0, 0, 0, -17, -65, -67] 0x000000efbfbd != 0x00000081

I'd need b[] the same array before and after encoding which it isn't (this referrers to the first answer).


Charset UTF8_CHARSET = Charset.forName("UTF-8");
String strISO = "{\"name\":\"?\"}";
System.out.println(strISO);
byte[] b = strISO.getBytes();
for (byte c: b) {
    System.out.print("[" + c + "]");
}
String str = new String(b, UTF8_CHARSET);
System.out.println(str);

As an alternative, StringUtils from Apache Commons can be used.

 byte[] bytes = {(byte) 1};
 String convertedString = StringUtils.newStringUtf8(bytes);

or

 String myString = "example";
 byte[] convertedBytes = StringUtils.getBytesUtf8(myString);

If you have non-standard charset, you can use getBytesUnchecked() or newString() accordingly.


I can't comment but don't want to start a new thread. But this isn't working. A simple round trip:

byte[] b = new byte[]{ 0, 0, 0, -127 };  // 0x00000081
String s = new String(b,StandardCharsets.UTF_8); // UTF8 = 0x0000, 0x0000,  0x0000, 0xfffd
b = s.getBytes(StandardCharsets.UTF_8); // [0, 0, 0, -17, -65, -67] 0x000000efbfbd != 0x00000081

I'd need b[] the same array before and after encoding which it isn't (this referrers to the first answer).


For decoding a series of bytes to a normal string message I finally got it working with UTF-8 encoding with this code:

/* Convert a list of UTF-8 numbers to a normal String
 * Usefull for decoding a jms message that is delivered as a sequence of bytes instead of plain text
 */
public String convertUtf8NumbersToString(String[] numbers){
    int length = numbers.length;
    byte[] data = new byte[length];

    for(int i = 0; i< length; i++){
        data[i] = Byte.parseByte(numbers[i]);
    }
    return new String(data, Charset.forName("UTF-8"));
}

My tomcat7 implementation is accepting strings as ISO-8859-1; despite the content-type of the HTTP request. The following solution worked for me when trying to correctly interpret characters like 'é' .

byte[] b1 = szP1.getBytes("ISO-8859-1");
System.out.println(b1.toString());

String szUT8 = new String(b1, "UTF-8");
System.out.println(szUT8);

When trying to interpret the string as US-ASCII, the byte info wasn't correctly interpreted.

b1 = szP1.getBytes("US-ASCII");
System.out.println(b1.toString());

//query is your json   

 DefaultHttpClient httpClient = new DefaultHttpClient();
 HttpPost postRequest = new HttpPost("http://my.site/test/v1/product/search?qy=");

 StringEntity input = new StringEntity(query, "UTF-8");
 input.setContentType("application/json");
 postRequest.setEntity(input);   
 HttpResponse response=response = httpClient.execute(postRequest);

String original = "hello world";
byte[] utf8Bytes = original.getBytes("UTF-8");

Examples related to java

Under what circumstances can I call findViewById with an Options Menu / Action Bar item? How much should a function trust another function How to implement a simple scenario the OO way Two constructors How do I get some variable from another class in Java? this in equals method How to split a string in two and store it in a field How to do perspective fixing? String index out of range: 4 My eclipse won't open, i download the bundle pack it keeps saying error log

Examples related to string

How to split a string in two and store it in a field String method cannot be found in a main class method Kotlin - How to correctly concatenate a String Replacing a character from a certain index Remove quotes from String in Python Detect whether a Python string is a number or a letter How does String substring work in Swift How does String.Index work in Swift swift 3.0 Data to String? How to parse JSON string in Typescript

Examples related to encoding

How to check encoding of a CSV file UnicodeEncodeError: 'ascii' codec can't encode character at special name Using Javascript's atob to decode base64 doesn't properly decode utf-8 strings What is the difference between utf8mb4 and utf8 charsets in MySQL? The character encoding of the plain text document was not declared - mootool script UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 23: ordinal not in range(128) How to encode text to base64 in python UTF-8 output from PowerShell Set Encoding of File to UTF8 With BOM in Sublime Text 3 Replace non-ASCII characters with a single space

Examples related to character-encoding

Changing PowerShell's default output encoding to UTF-8 JsonParseException : Illegal unquoted character ((CTRL-CHAR, code 10) Change the encoding of a file in Visual Studio Code What is the difference between utf8mb4 and utf8 charsets in MySQL? How to open html file? All inclusive Charset to avoid "java.nio.charset.MalformedInputException: Input length = 1"? UTF-8 output from PowerShell ERROR 1115 (42000): Unknown character set: 'utf8mb4' "for line in..." results in UnicodeDecodeError: 'utf-8' codec can't decode byte How to make php display \t \n as tab and new line instead of characters