First of all, if you are trying to encode apostophes for querystrings, they need to be URLEncoded, not escaped with a leading backslash. For that use URLEncoder.encode(String, String) (BTW: the second argument should always be "UTF-8"). Secondly, if you want to replace all instances of apostophe with backslash apostrophe, you must escape the backslash in your string expression with a leading backslash. Like this:
"This is' it".replace("'", "\\'");
I see now that you are probably trying to dynamically build a SQL statement. Do not do it this way. Your code will be susceptible to SQL injection attacks. Instead use a PreparedStatement.