Using BeautifulSoup to extract text without tags

Question

My webpage looks like this    lt p gt     lt strong class  offender  gt YOB  lt  strong gt  1987 lt br  gt     lt strong class  offender  gt RACE  lt  strong gt  WHITE lt br  gt     lt strong class  offender  gt GENDER  lt  strong gt  FEMALE lt br  gt     lt strong class  offender  gt HEIGHT  lt  strong gt  5 05   lt br  gt     lt strong class  offender  gt WEIGHT  lt  strong gt  118 lt br  gt     lt strong class  offender  gt EYE COLOR  lt  strong gt  GREEN lt br  gt     lt strong class  offender  gt HAIR COLOR  lt  strong gt  BROWN lt br  gt   lt  p gt    I want to extract the info for each individual and get YOB 1987  RACE WHITE  etc     What I tried is   subc   soup find all  p   subc1   subc 1  subc2   subc1 find all  strong     But this gives me only the values of YOB   RACE   etc     Is there a way that I can get the data in YOB 1987  RACE WHITE format

User · Answer

I think you could solve this with  strip   in gazpacho  Input  html    quot  quot  quot    lt p gt     lt strong class  quot offender quot  gt YOB  lt  strong gt  1987 lt br   gt     lt strong class  quot offender quot  gt RACE  lt  strong gt  WHITE lt br   gt     lt strong class  quot offender quot  gt GENDER  lt  strong gt  FEMALE lt br   gt     lt strong class  quot offender quot  gt HEIGHT  lt  strong gt  5 05   lt br   gt     lt strong class  quot offender quot  gt WEIGHT  lt  strong gt  118 lt br   gt     lt strong class  quot offender quot  gt EYE COLOR  lt  strong gt  GREEN lt br   gt     lt strong class  quot offender quot  gt HAIR COLOR  lt  strong gt  BROWN lt br   gt   lt  p gt   quot  quot  quot   Code  soup   Soup html  text   soup find  quot p quot   strip whitespace False    to keep  n characters intact lines         line strip       for line in text split  quot  n quot       if line     quot  quot    data   dict  line split  quot    quot   for line in lines    Output  print data      YOB    1987       RACE    WHITE       GENDER    FEMALE       HEIGHT    quot 5 05   quot       WEIGHT    118       EYE COLOR    GREEN       HAIR COLOR    BROWN

User · Answer

Just loop through all the  lt strong gt  tags and use next sibling to get what you want  Like this   for strong tag in soup find all  strong        print strong tag text  strong tag next sibling    Demo   from bs4 import BeautifulSoup  html        lt p gt     lt strong class  offender  gt YOB  lt  strong gt  1987 lt br   gt     lt strong class  offender  gt RACE  lt  strong gt  WHITE lt br   gt     lt strong class  offender  gt GENDER  lt  strong gt  FEMALE lt br   gt     lt strong class  offender  gt HEIGHT  lt  strong gt  5 05   lt br   gt     lt strong class  offender  gt WEIGHT  lt  strong gt  118 lt br   gt     lt strong class  offender  gt EYE COLOR  lt  strong gt  GREEN lt br   gt     lt strong class  offender  gt HAIR COLOR  lt  strong gt  BROWN lt br   gt   lt  p gt       soup   BeautifulSoup html   for strong tag in soup find all  strong        print strong tag text  strong tag next sibling    This gives you   YOB   1987 RACE   WHITE GENDER   FEMALE HEIGHT   5 05   WEIGHT   118 EYE COLOR   GREEN HAIR COLOR   BROWN

User · Answer

you can try this indside findall for loop  item price   item find  span   attrs   class   s-item  price    text  it extracts only text and assigs it to  quot item pice quot

User · Answer

I think you can get it using subc1 text    gt  gt  gt  html        lt p gt       lt strong class  offender  gt YOB  lt  strong gt  1987 lt br   gt       lt strong class  offender  gt RACE  lt  strong gt  WHITE lt br   gt       lt strong class  offender  gt GENDER  lt  strong gt  FEMALE lt br   gt       lt strong class  offender  gt HEIGHT  lt  strong gt  5 05   lt br   gt       lt strong class  offender  gt WEIGHT  lt  strong gt  118 lt br   gt       lt strong class  offender  gt EYE COLOR  lt  strong gt  GREEN lt br   gt       lt strong class  offender  gt HAIR COLOR  lt  strong gt  BROWN lt br   gt   lt  p gt       gt  gt  gt  from bs4 import BeautifulSoup  gt  gt  gt  soup   BeautifulSoup html   gt  gt  gt  print soup text   YOB  1987 RACE  WHITE GENDER  FEMALE HEIGHT  5 05   WEIGHT  118 EYE COLOR  GREEN HAIR COLOR  BROWN   Or if you want to explore it  you can use  contents     gt  gt  gt  p   soup find  p    gt  gt  gt  from pprint import pprint  gt  gt  gt  pprint p contents   u  n     lt strong class  offender  gt YOB  lt  strong gt    u  1987     lt br  gt    u  n     lt strong class  offender  gt RACE  lt  strong gt    u  WHITE     lt br  gt    u  n     lt strong class  offender  gt GENDER  lt  strong gt    u  FEMALE     lt br  gt    u  n     lt strong class  offender  gt HEIGHT  lt  strong gt    u  5 05       lt br  gt    u  n     lt strong class  offender  gt WEIGHT  lt  strong gt    u  118     lt br  gt    u  n     lt strong class  offender  gt EYE COLOR  lt  strong gt    u  GREEN     lt br  gt    u  n     lt strong class  offender  gt HAIR COLOR  lt  strong gt    u  BROWN     lt br  gt    u  n     and filter out the necessary items from the list    gt  gt  gt  data   dict zip  x text for x in p contents 1  4     x strip   for x in p contents 2  4      gt  gt  gt  pprint data   u EYE COLOR    u GREEN    u GENDER    u FEMALE    u HAIR COLOR    u BROWN    u HEIGHT    u 5 05      u RACE    u WHITE    u WEIGHT    u 118    u YOB    u 1987

[python] Using BeautifulSoup to extract text without tags

Examples related to python

Examples related to web-scraping

Examples related to beautifulsoup