[python] How do I verify that a string only contains letters, numbers, underscores and dashes?

I know how to do this if I iterate through all of the characters in the string but I am looking for a more elegant method.

This question is related to python regex string

The answer is


A regular expression will do the trick with very little code:

import re

...

if re.match("^[A-Za-z0-9_-]*$", my_little_string):
    # do something here

Regular expression can be very flexible.

import re;
re.fullmatch("^[\w-]+$", target_string) # fullmatch looks also workable for python 3.4

\w: Only [a-zA-Z0-9_]

So you need to add - char for justify hyphen char.

+: Match one or more repetitions of the preceding char. I guess you don't accept blank input. But if you do, change to * .

^: Matches the start of the string.

$: Matches the end of the string.

You need these two special characters since you need to avoid the following case. The unwanted chars like & here might appear between the matched pattern.

&&&PATTERN&&PATTERN


If it were not for the dashes and underscores, the easiest solution would be

my_little_string.isalnum()

(Section 3.6.1 of the Python Library Reference)


As an alternative to using regex you could do it in Sets:

from sets import Set

allowed_chars = Set('0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ_-')

if Set(my_little_sting).issubset(allowed_chars):
    # your action
    print True

There are a variety of ways of achieving this goal, some are clearer than others. For each of my examples, 'True' means that the string passed is valid, 'False' means it contains invalid characters.

First of all, there's the naive approach:

import string
allowed = string.letters + string.digits + '_' + '-'

def check_naive(mystring):
    return all(c in allowed for c in mystring)

Then there's use of a regular expression, you can do this with re.match(). Note that '-' has to be at the end of the [] otherwise it will be used as a 'range' delimiter. Also note the $ which means 'end of string'. Other answers noted in this question use a special character class, '\w', I always prefer using an explicit character class range using [] because it is easier to understand without having to look up a quick reference guide, and easier to special-case.

import re
CHECK_RE = re.compile('[a-zA-Z0-9_-]+$')
def check_re(mystring):
    return CHECK_RE.match(mystring)

Another solution noted that you can do an inverse match with regular expressions, I've included that here now. Note that [^...] inverts the character class because the ^ is used:

CHECK_INV_RE = re.compile('[^a-zA-Z0-9_-]')
def check_inv_re(mystring):
   return not CHECK_INV_RE.search(mystring)

You can also do something tricky with the 'set' object. Have a look at this example, which removes from the original string all the characters that are allowed, leaving us with a set containing either a) nothing, or b) the offending characters from the string:

def check_set(mystring):
    return not set(mystring) - set(allowed)

 pat = re.compile ('[^\w-]')

 def onlyallowed(s):
    return not pat.search (s)

[Edit] There's another solution not mentioned yet, and it seems to outperform the others given so far in most cases.

Use string.translate to replace all valid characters in the string, and see if we have any invalid ones left over. This is pretty fast as it uses the underlying C function to do the work, with very little python bytecode involved.

Obviously performance isn't everything - going for the most readable solutions is probably the best approach when not in a performance critical codepath, but just to see how the solutions stack up, here's a performance comparison of all the methods proposed so far. check_trans is the one using the string.translate method.

Test code:

import string, re, timeit

pat = re.compile('[\w-]*$')
pat_inv = re.compile ('[^\w-]')
allowed_chars=string.ascii_letters + string.digits + '_-'
allowed_set = set(allowed_chars)
trans_table = string.maketrans('','')

def check_set_diff(s):
    return not set(s) - allowed_set

def check_set_all(s):
    return all(x in allowed_set for x in s)

def check_set_subset(s):
    return set(s).issubset(allowed_set)

def check_re_match(s):
    return pat.match(s)

def check_re_inverse(s): # Search for non-matching character.
    return not pat_inv.search(s)

def check_trans(s):
    return not s.translate(trans_table,allowed_chars)

test_long_almost_valid='a_very_long_string_that_is_mostly_valid_except_for_last_char'*99 + '!'
test_long_valid='a_very_long_string_that_is_completely_valid_' * 99
test_short_valid='short_valid_string'
test_short_invalid='/$%$%&'
test_long_invalid='/$%$%&' * 99
test_empty=''

def main():
    funcs = sorted(f for f in globals() if f.startswith('check_'))
    tests = sorted(f for f in globals() if f.startswith('test_'))
    for test in tests:
        print "Test %-15s (length = %d):" % (test, len(globals()[test]))
        for func in funcs:
            print "  %-20s : %.3f" % (func, 
                   timeit.Timer('%s(%s)' % (func, test), 'from __main__ import pat,allowed_set,%s' % ','.join(funcs+tests)).timeit(10000))
        print

if __name__=='__main__': main()

The results on my system are:

Test test_empty      (length = 0):
  check_re_inverse     : 0.042
  check_re_match       : 0.030
  check_set_all        : 0.027
  check_set_diff       : 0.029
  check_set_subset     : 0.029
  check_trans          : 0.014

Test test_long_almost_valid (length = 5941):
  check_re_inverse     : 2.690
  check_re_match       : 3.037
  check_set_all        : 18.860
  check_set_diff       : 2.905
  check_set_subset     : 2.903
  check_trans          : 0.182

Test test_long_invalid (length = 594):
  check_re_inverse     : 0.017
  check_re_match       : 0.015
  check_set_all        : 0.044
  check_set_diff       : 0.311
  check_set_subset     : 0.308
  check_trans          : 0.034

Test test_long_valid (length = 4356):
  check_re_inverse     : 1.890
  check_re_match       : 1.010
  check_set_all        : 14.411
  check_set_diff       : 2.101
  check_set_subset     : 2.333
  check_trans          : 0.140

Test test_short_invalid (length = 6):
  check_re_inverse     : 0.017
  check_re_match       : 0.019
  check_set_all        : 0.044
  check_set_diff       : 0.032
  check_set_subset     : 0.037
  check_trans          : 0.015

Test test_short_valid (length = 18):
  check_re_inverse     : 0.125
  check_re_match       : 0.066
  check_set_all        : 0.104
  check_set_diff       : 0.051
  check_set_subset     : 0.046
  check_trans          : 0.017

The translate approach seems best in most cases, dramatically so with long valid strings, but is beaten out by regexes in test_long_invalid (Presumably because the regex can bail out immediately, but translate always has to scan the whole string). The set approaches are usually worst, beating regexes only for the empty string case.

Using all(x in allowed_set for x in s) performs well if it bails out early, but can be bad if it has to iterate through every character. isSubSet and set difference are comparable, and are consistently proportional to the length of the string regardless of the data.

There's a similar difference between the regex methods matching all valid characters and searching for invalid characters. Matching performs a little better when checking for a long, but fully valid string, but worse for invalid characters near the end of the string.


If it were not for the dashes and underscores, the easiest solution would be

my_little_string.isalnum()

(Section 3.6.1 of the Python Library Reference)


Regular expression can be very flexible.

import re;
re.fullmatch("^[\w-]+$", target_string) # fullmatch looks also workable for python 3.4

\w: Only [a-zA-Z0-9_]

So you need to add - char for justify hyphen char.

+: Match one or more repetitions of the preceding char. I guess you don't accept blank input. But if you do, change to * .

^: Matches the start of the string.

$: Matches the end of the string.

You need these two special characters since you need to avoid the following case. The unwanted chars like & here might appear between the matched pattern.

&&&PATTERN&&PATTERN


There are a variety of ways of achieving this goal, some are clearer than others. For each of my examples, 'True' means that the string passed is valid, 'False' means it contains invalid characters.

First of all, there's the naive approach:

import string
allowed = string.letters + string.digits + '_' + '-'

def check_naive(mystring):
    return all(c in allowed for c in mystring)

Then there's use of a regular expression, you can do this with re.match(). Note that '-' has to be at the end of the [] otherwise it will be used as a 'range' delimiter. Also note the $ which means 'end of string'. Other answers noted in this question use a special character class, '\w', I always prefer using an explicit character class range using [] because it is easier to understand without having to look up a quick reference guide, and easier to special-case.

import re
CHECK_RE = re.compile('[a-zA-Z0-9_-]+$')
def check_re(mystring):
    return CHECK_RE.match(mystring)

Another solution noted that you can do an inverse match with regular expressions, I've included that here now. Note that [^...] inverts the character class because the ^ is used:

CHECK_INV_RE = re.compile('[^a-zA-Z0-9_-]')
def check_inv_re(mystring):
   return not CHECK_INV_RE.search(mystring)

You can also do something tricky with the 'set' object. Have a look at this example, which removes from the original string all the characters that are allowed, leaving us with a set containing either a) nothing, or b) the offending characters from the string:

def check_set(mystring):
    return not set(mystring) - set(allowed)

use a regex and see if it matches!

([a-z][A-Z][0-9]\_\-)*

Here's something based on Jerub's "naive approach" (naive being his words, not mine!):

import string
ALLOWED = frozenset(string.ascii_letters + string.digits + '_' + '-')

def check(mystring):
    return all(c in ALLOWED for c in mystring)

If ALLOWED was a string then I think c in ALLOWED would involve iterating over each character in the string until it found a match or reached the end. Which, to quote Joel Spolsky, is something of a Shlemiel the Painter algorithm.

But testing for existence in a set should be more efficient, or at least less dependent on the number of allowed characters. Certainly this approach is a little bit faster on my machine. It's clear and I think it performs plenty well enough for most cases (on my slow machine I can validate tens of thousands of short-ish strings in a fraction of a second). I like it.

ACTUALLY on my machine a regexp works out several times faster, and is just as simple as this (arguably simpler). So that probably is the best way forward.


[Edit] There's another solution not mentioned yet, and it seems to outperform the others given so far in most cases.

Use string.translate to replace all valid characters in the string, and see if we have any invalid ones left over. This is pretty fast as it uses the underlying C function to do the work, with very little python bytecode involved.

Obviously performance isn't everything - going for the most readable solutions is probably the best approach when not in a performance critical codepath, but just to see how the solutions stack up, here's a performance comparison of all the methods proposed so far. check_trans is the one using the string.translate method.

Test code:

import string, re, timeit

pat = re.compile('[\w-]*$')
pat_inv = re.compile ('[^\w-]')
allowed_chars=string.ascii_letters + string.digits + '_-'
allowed_set = set(allowed_chars)
trans_table = string.maketrans('','')

def check_set_diff(s):
    return not set(s) - allowed_set

def check_set_all(s):
    return all(x in allowed_set for x in s)

def check_set_subset(s):
    return set(s).issubset(allowed_set)

def check_re_match(s):
    return pat.match(s)

def check_re_inverse(s): # Search for non-matching character.
    return not pat_inv.search(s)

def check_trans(s):
    return not s.translate(trans_table,allowed_chars)

test_long_almost_valid='a_very_long_string_that_is_mostly_valid_except_for_last_char'*99 + '!'
test_long_valid='a_very_long_string_that_is_completely_valid_' * 99
test_short_valid='short_valid_string'
test_short_invalid='/$%$%&'
test_long_invalid='/$%$%&' * 99
test_empty=''

def main():
    funcs = sorted(f for f in globals() if f.startswith('check_'))
    tests = sorted(f for f in globals() if f.startswith('test_'))
    for test in tests:
        print "Test %-15s (length = %d):" % (test, len(globals()[test]))
        for func in funcs:
            print "  %-20s : %.3f" % (func, 
                   timeit.Timer('%s(%s)' % (func, test), 'from __main__ import pat,allowed_set,%s' % ','.join(funcs+tests)).timeit(10000))
        print

if __name__=='__main__': main()

The results on my system are:

Test test_empty      (length = 0):
  check_re_inverse     : 0.042
  check_re_match       : 0.030
  check_set_all        : 0.027
  check_set_diff       : 0.029
  check_set_subset     : 0.029
  check_trans          : 0.014

Test test_long_almost_valid (length = 5941):
  check_re_inverse     : 2.690
  check_re_match       : 3.037
  check_set_all        : 18.860
  check_set_diff       : 2.905
  check_set_subset     : 2.903
  check_trans          : 0.182

Test test_long_invalid (length = 594):
  check_re_inverse     : 0.017
  check_re_match       : 0.015
  check_set_all        : 0.044
  check_set_diff       : 0.311
  check_set_subset     : 0.308
  check_trans          : 0.034

Test test_long_valid (length = 4356):
  check_re_inverse     : 1.890
  check_re_match       : 1.010
  check_set_all        : 14.411
  check_set_diff       : 2.101
  check_set_subset     : 2.333
  check_trans          : 0.140

Test test_short_invalid (length = 6):
  check_re_inverse     : 0.017
  check_re_match       : 0.019
  check_set_all        : 0.044
  check_set_diff       : 0.032
  check_set_subset     : 0.037
  check_trans          : 0.015

Test test_short_valid (length = 18):
  check_re_inverse     : 0.125
  check_re_match       : 0.066
  check_set_all        : 0.104
  check_set_diff       : 0.051
  check_set_subset     : 0.046
  check_trans          : 0.017

The translate approach seems best in most cases, dramatically so with long valid strings, but is beaten out by regexes in test_long_invalid (Presumably because the regex can bail out immediately, but translate always has to scan the whole string). The set approaches are usually worst, beating regexes only for the empty string case.

Using all(x in allowed_set for x in s) performs well if it bails out early, but can be bad if it has to iterate through every character. isSubSet and set difference are comparable, and are consistently proportional to the length of the string regardless of the data.

There's a similar difference between the regex methods matching all valid characters and searching for invalid characters. Matching performs a little better when checking for a long, but fully valid string, but worse for invalid characters near the end of the string.


Well you can ask the help of regex, the great in here :)

code:

import re

string = 'adsfg34wrtwe4r2_()' #your string that needs to be matched.
regex = r'^[\w\d_()]*$' # you can also add a space in regex if u want to allow it in the string  
if re.match(regex,string):
    print 'yes'
else: 
    print 'false'

Output:

yes  

Hope this helps :)


You could always use a list comprehension and check the results with all, it would be a little less resource intensive than using a regex: all([c in string.letters + string.digits + ["_", "-"] for c in mystring])


 pat = re.compile ('[^\w-]')

 def onlyallowed(s):
    return not pat.search (s)

Here's something based on Jerub's "naive approach" (naive being his words, not mine!):

import string
ALLOWED = frozenset(string.ascii_letters + string.digits + '_' + '-')

def check(mystring):
    return all(c in ALLOWED for c in mystring)

If ALLOWED was a string then I think c in ALLOWED would involve iterating over each character in the string until it found a match or reached the end. Which, to quote Joel Spolsky, is something of a Shlemiel the Painter algorithm.

But testing for existence in a set should be more efficient, or at least less dependent on the number of allowed characters. Certainly this approach is a little bit faster on my machine. It's clear and I think it performs plenty well enough for most cases (on my slow machine I can validate tens of thousands of short-ish strings in a fraction of a second). I like it.

ACTUALLY on my machine a regexp works out several times faster, and is just as simple as this (arguably simpler). So that probably is the best way forward.


If it were not for the dashes and underscores, the easiest solution would be

my_little_string.isalnum()

(Section 3.6.1 of the Python Library Reference)


You could always use a list comprehension and check the results with all, it would be a little less resource intensive than using a regex: all([c in string.letters + string.digits + ["_", "-"] for c in mystring])


[Edit] There's another solution not mentioned yet, and it seems to outperform the others given so far in most cases.

Use string.translate to replace all valid characters in the string, and see if we have any invalid ones left over. This is pretty fast as it uses the underlying C function to do the work, with very little python bytecode involved.

Obviously performance isn't everything - going for the most readable solutions is probably the best approach when not in a performance critical codepath, but just to see how the solutions stack up, here's a performance comparison of all the methods proposed so far. check_trans is the one using the string.translate method.

Test code:

import string, re, timeit

pat = re.compile('[\w-]*$')
pat_inv = re.compile ('[^\w-]')
allowed_chars=string.ascii_letters + string.digits + '_-'
allowed_set = set(allowed_chars)
trans_table = string.maketrans('','')

def check_set_diff(s):
    return not set(s) - allowed_set

def check_set_all(s):
    return all(x in allowed_set for x in s)

def check_set_subset(s):
    return set(s).issubset(allowed_set)

def check_re_match(s):
    return pat.match(s)

def check_re_inverse(s): # Search for non-matching character.
    return not pat_inv.search(s)

def check_trans(s):
    return not s.translate(trans_table,allowed_chars)

test_long_almost_valid='a_very_long_string_that_is_mostly_valid_except_for_last_char'*99 + '!'
test_long_valid='a_very_long_string_that_is_completely_valid_' * 99
test_short_valid='short_valid_string'
test_short_invalid='/$%$%&'
test_long_invalid='/$%$%&' * 99
test_empty=''

def main():
    funcs = sorted(f for f in globals() if f.startswith('check_'))
    tests = sorted(f for f in globals() if f.startswith('test_'))
    for test in tests:
        print "Test %-15s (length = %d):" % (test, len(globals()[test]))
        for func in funcs:
            print "  %-20s : %.3f" % (func, 
                   timeit.Timer('%s(%s)' % (func, test), 'from __main__ import pat,allowed_set,%s' % ','.join(funcs+tests)).timeit(10000))
        print

if __name__=='__main__': main()

The results on my system are:

Test test_empty      (length = 0):
  check_re_inverse     : 0.042
  check_re_match       : 0.030
  check_set_all        : 0.027
  check_set_diff       : 0.029
  check_set_subset     : 0.029
  check_trans          : 0.014

Test test_long_almost_valid (length = 5941):
  check_re_inverse     : 2.690
  check_re_match       : 3.037
  check_set_all        : 18.860
  check_set_diff       : 2.905
  check_set_subset     : 2.903
  check_trans          : 0.182

Test test_long_invalid (length = 594):
  check_re_inverse     : 0.017
  check_re_match       : 0.015
  check_set_all        : 0.044
  check_set_diff       : 0.311
  check_set_subset     : 0.308
  check_trans          : 0.034

Test test_long_valid (length = 4356):
  check_re_inverse     : 1.890
  check_re_match       : 1.010
  check_set_all        : 14.411
  check_set_diff       : 2.101
  check_set_subset     : 2.333
  check_trans          : 0.140

Test test_short_invalid (length = 6):
  check_re_inverse     : 0.017
  check_re_match       : 0.019
  check_set_all        : 0.044
  check_set_diff       : 0.032
  check_set_subset     : 0.037
  check_trans          : 0.015

Test test_short_valid (length = 18):
  check_re_inverse     : 0.125
  check_re_match       : 0.066
  check_set_all        : 0.104
  check_set_diff       : 0.051
  check_set_subset     : 0.046
  check_trans          : 0.017

The translate approach seems best in most cases, dramatically so with long valid strings, but is beaten out by regexes in test_long_invalid (Presumably because the regex can bail out immediately, but translate always has to scan the whole string). The set approaches are usually worst, beating regexes only for the empty string case.

Using all(x in allowed_set for x in s) performs well if it bails out early, but can be bad if it has to iterate through every character. isSubSet and set difference are comparable, and are consistently proportional to the length of the string regardless of the data.

There's a similar difference between the regex methods matching all valid characters and searching for invalid characters. Matching performs a little better when checking for a long, but fully valid string, but worse for invalid characters near the end of the string.


use a regex and see if it matches!

([a-z][A-Z][0-9]\_\-)*

As an alternative to using regex you could do it in Sets:

from sets import Set

allowed_chars = Set('0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ_-')

if Set(my_little_sting).issubset(allowed_chars):
    # your action
    print True

If it were not for the dashes and underscores, the easiest solution would be

my_little_string.isalnum()

(Section 3.6.1 of the Python Library Reference)


 pat = re.compile ('[^\w-]')

 def onlyallowed(s):
    return not pat.search (s)

As an alternative to using regex you could do it in Sets:

from sets import Set

allowed_chars = Set('0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ_-')

if Set(my_little_sting).issubset(allowed_chars):
    # your action
    print True

use a regex and see if it matches!

([a-z][A-Z][0-9]\_\-)*

There are a variety of ways of achieving this goal, some are clearer than others. For each of my examples, 'True' means that the string passed is valid, 'False' means it contains invalid characters.

First of all, there's the naive approach:

import string
allowed = string.letters + string.digits + '_' + '-'

def check_naive(mystring):
    return all(c in allowed for c in mystring)

Then there's use of a regular expression, you can do this with re.match(). Note that '-' has to be at the end of the [] otherwise it will be used as a 'range' delimiter. Also note the $ which means 'end of string'. Other answers noted in this question use a special character class, '\w', I always prefer using an explicit character class range using [] because it is easier to understand without having to look up a quick reference guide, and easier to special-case.

import re
CHECK_RE = re.compile('[a-zA-Z0-9_-]+$')
def check_re(mystring):
    return CHECK_RE.match(mystring)

Another solution noted that you can do an inverse match with regular expressions, I've included that here now. Note that [^...] inverts the character class because the ^ is used:

CHECK_INV_RE = re.compile('[^a-zA-Z0-9_-]')
def check_inv_re(mystring):
   return not CHECK_INV_RE.search(mystring)

You can also do something tricky with the 'set' object. Have a look at this example, which removes from the original string all the characters that are allowed, leaving us with a set containing either a) nothing, or b) the offending characters from the string:

def check_set(mystring):
    return not set(mystring) - set(allowed)

As an alternative to using regex you could do it in Sets:

from sets import Set

allowed_chars = Set('0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ_-')

if Set(my_little_sting).issubset(allowed_chars):
    # your action
    print True

 pat = re.compile ('[^\w-]')

 def onlyallowed(s):
    return not pat.search (s)

There are a variety of ways of achieving this goal, some are clearer than others. For each of my examples, 'True' means that the string passed is valid, 'False' means it contains invalid characters.

First of all, there's the naive approach:

import string
allowed = string.letters + string.digits + '_' + '-'

def check_naive(mystring):
    return all(c in allowed for c in mystring)

Then there's use of a regular expression, you can do this with re.match(). Note that '-' has to be at the end of the [] otherwise it will be used as a 'range' delimiter. Also note the $ which means 'end of string'. Other answers noted in this question use a special character class, '\w', I always prefer using an explicit character class range using [] because it is easier to understand without having to look up a quick reference guide, and easier to special-case.

import re
CHECK_RE = re.compile('[a-zA-Z0-9_-]+$')
def check_re(mystring):
    return CHECK_RE.match(mystring)

Another solution noted that you can do an inverse match with regular expressions, I've included that here now. Note that [^...] inverts the character class because the ^ is used:

CHECK_INV_RE = re.compile('[^a-zA-Z0-9_-]')
def check_inv_re(mystring):
   return not CHECK_INV_RE.search(mystring)

You can also do something tricky with the 'set' object. Have a look at this example, which removes from the original string all the characters that are allowed, leaving us with a set containing either a) nothing, or b) the offending characters from the string:

def check_set(mystring):
    return not set(mystring) - set(allowed)

You could always use a list comprehension and check the results with all, it would be a little less resource intensive than using a regex: all([c in string.letters + string.digits + ["_", "-"] for c in mystring])


Well you can ask the help of regex, the great in here :)

code:

import re

string = 'adsfg34wrtwe4r2_()' #your string that needs to be matched.
regex = r'^[\w\d_()]*$' # you can also add a space in regex if u want to allow it in the string  
if re.match(regex,string):
    print 'yes'
else: 
    print 'false'

Output:

yes  

Hope this helps :)


[Edit] There's another solution not mentioned yet, and it seems to outperform the others given so far in most cases.

Use string.translate to replace all valid characters in the string, and see if we have any invalid ones left over. This is pretty fast as it uses the underlying C function to do the work, with very little python bytecode involved.

Obviously performance isn't everything - going for the most readable solutions is probably the best approach when not in a performance critical codepath, but just to see how the solutions stack up, here's a performance comparison of all the methods proposed so far. check_trans is the one using the string.translate method.

Test code:

import string, re, timeit

pat = re.compile('[\w-]*$')
pat_inv = re.compile ('[^\w-]')
allowed_chars=string.ascii_letters + string.digits + '_-'
allowed_set = set(allowed_chars)
trans_table = string.maketrans('','')

def check_set_diff(s):
    return not set(s) - allowed_set

def check_set_all(s):
    return all(x in allowed_set for x in s)

def check_set_subset(s):
    return set(s).issubset(allowed_set)

def check_re_match(s):
    return pat.match(s)

def check_re_inverse(s): # Search for non-matching character.
    return not pat_inv.search(s)

def check_trans(s):
    return not s.translate(trans_table,allowed_chars)

test_long_almost_valid='a_very_long_string_that_is_mostly_valid_except_for_last_char'*99 + '!'
test_long_valid='a_very_long_string_that_is_completely_valid_' * 99
test_short_valid='short_valid_string'
test_short_invalid='/$%$%&'
test_long_invalid='/$%$%&' * 99
test_empty=''

def main():
    funcs = sorted(f for f in globals() if f.startswith('check_'))
    tests = sorted(f for f in globals() if f.startswith('test_'))
    for test in tests:
        print "Test %-15s (length = %d):" % (test, len(globals()[test]))
        for func in funcs:
            print "  %-20s : %.3f" % (func, 
                   timeit.Timer('%s(%s)' % (func, test), 'from __main__ import pat,allowed_set,%s' % ','.join(funcs+tests)).timeit(10000))
        print

if __name__=='__main__': main()

The results on my system are:

Test test_empty      (length = 0):
  check_re_inverse     : 0.042
  check_re_match       : 0.030
  check_set_all        : 0.027
  check_set_diff       : 0.029
  check_set_subset     : 0.029
  check_trans          : 0.014

Test test_long_almost_valid (length = 5941):
  check_re_inverse     : 2.690
  check_re_match       : 3.037
  check_set_all        : 18.860
  check_set_diff       : 2.905
  check_set_subset     : 2.903
  check_trans          : 0.182

Test test_long_invalid (length = 594):
  check_re_inverse     : 0.017
  check_re_match       : 0.015
  check_set_all        : 0.044
  check_set_diff       : 0.311
  check_set_subset     : 0.308
  check_trans          : 0.034

Test test_long_valid (length = 4356):
  check_re_inverse     : 1.890
  check_re_match       : 1.010
  check_set_all        : 14.411
  check_set_diff       : 2.101
  check_set_subset     : 2.333
  check_trans          : 0.140

Test test_short_invalid (length = 6):
  check_re_inverse     : 0.017
  check_re_match       : 0.019
  check_set_all        : 0.044
  check_set_diff       : 0.032
  check_set_subset     : 0.037
  check_trans          : 0.015

Test test_short_valid (length = 18):
  check_re_inverse     : 0.125
  check_re_match       : 0.066
  check_set_all        : 0.104
  check_set_diff       : 0.051
  check_set_subset     : 0.046
  check_trans          : 0.017

The translate approach seems best in most cases, dramatically so with long valid strings, but is beaten out by regexes in test_long_invalid (Presumably because the regex can bail out immediately, but translate always has to scan the whole string). The set approaches are usually worst, beating regexes only for the empty string case.

Using all(x in allowed_set for x in s) performs well if it bails out early, but can be bad if it has to iterate through every character. isSubSet and set difference are comparable, and are consistently proportional to the length of the string regardless of the data.

There's a similar difference between the regex methods matching all valid characters and searching for invalid characters. Matching performs a little better when checking for a long, but fully valid string, but worse for invalid characters near the end of the string.


Examples related to python

programming a servo thru a barometer Is there a way to view two blocks of code from the same file simultaneously in Sublime Text? python variable NameError Why my regexp for hyphenated words doesn't work? Comparing a variable with a string python not working when redirecting from bash script is it possible to add colors to python output? Get Public URL for File - Google Cloud Storage - App Engine (Python) Real time face detection OpenCV, Python xlrd.biffh.XLRDError: Excel xlsx file; not supported Could not load dynamic library 'cudart64_101.dll' on tensorflow CPU-only installation

Examples related to regex

Why my regexp for hyphenated words doesn't work? grep's at sign caught as whitespace Preg_match backtrack error regex match any single character (one character only) re.sub erroring with "Expected string or bytes-like object" Only numbers. Input number in React Visual Studio Code Search and Replace with Regular Expressions Strip / trim all strings of a dataframe return string with first match Regex How to capture multiple repeated groups?

Examples related to string

How to split a string in two and store it in a field String method cannot be found in a main class method Kotlin - How to correctly concatenate a String Replacing a character from a certain index Remove quotes from String in Python Detect whether a Python string is a number or a letter How does String substring work in Swift How does String.Index work in Swift swift 3.0 Data to String? How to parse JSON string in Typescript