What is the best way to strip all non alphanumeric characters from a string, using Python?
The solutions presented in the PHP variant of this question will probably work with some minor adjustments, but don't seem very 'pythonic' to me.
For the record, I don't just want to strip periods and commas (and other punctuation), but also quotes, brackets, etc.
This question is related to
python
string
non-alphanumeric
How about:
def ExtractAlphanumeric(InputString):
from string import ascii_letters, digits
return "".join([ch for ch in InputString if ch in (ascii_letters + digits)])
This works by using list comprehension to produce a list of the characters in InputString if they are present in the combined ascii_letters and digits strings. It then joins the list together into a string.
Regular expressions to the rescue:
import re
re.sub(r'\W+', '', your_string)
By Python definition
'\W==[^a-zA-Z0-9_], which excludes allnumbers,lettersand_
>>> import re
>>> string = "Kl13@£$%[};'\""
>>> pattern = re.compile('\W')
>>> string = re.sub(pattern, '', string)
>>> print string
Kl13
sent = "".join(e for e in sent if e.isalpha())
for char in my_string:
if not char.isalnum():
my_string = my_string.replace(char,"")
You could try:
print ''.join(ch for ch in some_string if ch.isalnum())
Timing with random strings of ASCII printables:
from inspect import getsource
from random import sample
import re
from string import printable
from timeit import timeit
pattern_single = re.compile(r'[\W]')
pattern_repeat = re.compile(r'[\W]+')
translation_tb = str.maketrans('', '', ''.join(c for c in map(chr, range(256)) if not c.isalnum()))
def generate_test_string(length):
return ''.join(sample(printable, length))
def main():
for i in range(0, 60, 10):
for test in [
lambda: ''.join(c for c in generate_test_string(i) if c.isalnum()),
lambda: ''.join(filter(str.isalnum, generate_test_string(i))),
lambda: re.sub(r'[\W]', '', generate_test_string(i)),
lambda: re.sub(r'[\W]+', '', generate_test_string(i)),
lambda: pattern_single.sub('', generate_test_string(i)),
lambda: pattern_repeat.sub('', generate_test_string(i)),
lambda: generate_test_string(i).translate(translation_tb),
]:
print(timeit(test), i, getsource(test).lstrip(' lambda: ').rstrip(',\n'), sep='\t')
if __name__ == '__main__':
main()
Result (Python 3.7):
Time Length Code
6.3716264850008880 00 ''.join(c for c in generate_test_string(i) if c.isalnum())
5.7285426190064750 00 ''.join(filter(str.isalnum, generate_test_string(i)))
8.1875841680011940 00 re.sub(r'[\W]', '', generate_test_string(i))
8.0002205439959650 00 re.sub(r'[\W]+', '', generate_test_string(i))
5.5290945199958510 00 pattern_single.sub('', generate_test_string(i))
5.4417179649972240 00 pattern_repeat.sub('', generate_test_string(i))
4.6772285089973590 00 generate_test_string(i).translate(translation_tb)
23.574712151996210 10 ''.join(c for c in generate_test_string(i) if c.isalnum())
22.829975890002970 10 ''.join(filter(str.isalnum, generate_test_string(i)))
27.210196289997840 10 re.sub(r'[\W]', '', generate_test_string(i))
27.203713296003116 10 re.sub(r'[\W]+', '', generate_test_string(i))
24.008979928999906 10 pattern_single.sub('', generate_test_string(i))
23.945240008994006 10 pattern_repeat.sub('', generate_test_string(i))
21.830899796994345 10 generate_test_string(i).translate(translation_tb)
38.731336012999236 20 ''.join(c for c in generate_test_string(i) if c.isalnum())
37.942474347000825 20 ''.join(filter(str.isalnum, generate_test_string(i)))
42.169366310001350 20 re.sub(r'[\W]', '', generate_test_string(i))
41.933375883003464 20 re.sub(r'[\W]+', '', generate_test_string(i))
38.899814646996674 20 pattern_single.sub('', generate_test_string(i))
38.636144253003295 20 pattern_repeat.sub('', generate_test_string(i))
36.201238164998360 20 generate_test_string(i).translate(translation_tb)
49.377356811004574 30 ''.join(c for c in generate_test_string(i) if c.isalnum())
48.408927293996385 30 ''.join(filter(str.isalnum, generate_test_string(i)))
53.901889764994850 30 re.sub(r'[\W]', '', generate_test_string(i))
52.130339455994545 30 re.sub(r'[\W]+', '', generate_test_string(i))
50.061149017004940 30 pattern_single.sub('', generate_test_string(i))
49.366573111998150 30 pattern_repeat.sub('', generate_test_string(i))
46.649754120997386 30 generate_test_string(i).translate(translation_tb)
63.107938601999194 40 ''.join(c for c in generate_test_string(i) if c.isalnum())
65.116287978999030 40 ''.join(filter(str.isalnum, generate_test_string(i)))
71.477421126997800 40 re.sub(r'[\W]', '', generate_test_string(i))
66.027950693998720 40 re.sub(r'[\W]+', '', generate_test_string(i))
63.315361931003280 40 pattern_single.sub('', generate_test_string(i))
62.342320287003530 40 pattern_repeat.sub('', generate_test_string(i))
58.249303059004890 40 generate_test_string(i).translate(translation_tb)
73.810345625002810 50 ''.join(c for c in generate_test_string(i) if c.isalnum())
72.593953348005020 50 ''.join(filter(str.isalnum, generate_test_string(i)))
76.048324580995540 50 re.sub(r'[\W]', '', generate_test_string(i))
75.106637657001560 50 re.sub(r'[\W]+', '', generate_test_string(i))
74.681338128997600 50 pattern_single.sub('', generate_test_string(i))
72.430461594005460 50 pattern_repeat.sub('', generate_test_string(i))
69.394243567003290 50 generate_test_string(i).translate(translation_tb)
str.maketrans & str.translate is fastest, but includes all non-ASCII characters.
re.compile & pattern.sub is slower, but is somehow faster than ''.join & filter.
As a spin off from some other answers here, I offer a really simple and flexible way to define a set of characters that you want to limit a string's content to. In this case, I'm allowing alphanumerics PLUS dash and underscore. Just add or remove characters from my PERMITTED_CHARS as suits your use case.
PERMITTED_CHARS = "0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ_-"
someString = "".join(c for c in someString if c in PERMITTED_CHARS)
If i understood correctly the easiest way is to use regular expression as it provides you lots of flexibility but the other simple method is to use for loop following is the code with example I also counted the occurrence of word and stored in dictionary..
s = """An... essay is, generally, a piece of writing that gives the author's own
argument — but the definition is vague,
overlapping with those of a paper, an article, a pamphlet, and a short story. Essays
have traditionally been
sub-classified as formal and informal. Formal essays are characterized by "serious
purpose, dignity, logical
organization, length," whereas the informal essay is characterized by "the personal
element (self-revelation,
individual tastes and experiences, confidential manner), humor, graceful style,
rambling structure, unconventionality
or novelty of theme," etc.[1]"""
d = {} # creating empty dic
words = s.split() # spliting string and stroing in list
for word in words:
new_word = ''
for c in word:
if c.isalnum(): # checking if indiviual chr is alphanumeric or not
new_word = new_word + c
print(new_word, end=' ')
# if new_word not in d:
# d[new_word] = 1
# else:
# d[new_word] = d[new_word] +1
print(d)
please rate this if this answer is useful!
Use the str.translate() method.
Presuming you will be doing this often:
(1) Once, create a string containing all the characters you wish to delete:
delchars = ''.join(c for c in map(chr, range(256)) if not c.isalnum())
(2) Whenever you want to scrunch a string:
scrunched = s.translate(None, delchars)
The setup cost probably compares favourably with re.compile; the marginal cost is way lower:
C:\junk>\python26\python -mtimeit -s"import string;d=''.join(c for c in map(chr,range(256)) if not c.isalnum());s=string.printable" "s.translate(None,d)"
100000 loops, best of 3: 2.04 usec per loop
C:\junk>\python26\python -mtimeit -s"import re,string;s=string.printable;r=re.compile(r'[\W_]+')" "r.sub('',s)"
100000 loops, best of 3: 7.34 usec per loop
Note: Using string.printable as benchmark data gives the pattern '[\W_]+' an unfair advantage; all the non-alphanumeric characters are in one bunch ... in typical data there would be more than one substitution to do:
C:\junk>\python26\python -c "import string; s = string.printable; print len(s),repr(s)"
100 '0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ!"#$%&\'()*+,-./:;=>?@[\\]^_`{|}~ \t\n\r\x0b\x0c'
Here's what happens if you give re.sub a bit more work to do:
C:\junk>\python26\python -mtimeit -s"d=''.join(c for c in map(chr,range(256)) if not c.isalnum());s='foo-'*25" "s.translate(None,d)"
1000000 loops, best of 3: 1.97 usec per loop
C:\junk>\python26\python -mtimeit -s"import re;s='foo-'*25;r=re.compile(r'[\W_]+')" "r.sub('',s)"
10000 loops, best of 3: 26.4 usec per loop
Source: Stackoverflow.com