Map HTML to JSON

Question

I m attempting map HTML into JSON with structure intact  Are there any libraries out there that do this or will I need to write my own  I suppose if there are no html2json libraries out there I could take an xml2json library as a start  After all  html is only a variant of xml anyway right   UPDATE  Okay  I should probably give an example  What I m trying to do is the following  Parse a string of html    lt div gt     lt span gt text lt  span gt Text2  lt  div gt    into a json object like so        type     div      content                   type     span          content               Text2                      Text2          NOTE  In case you didn t notice the tag  I m looking for a solution in Javascript

User · Answer

I got few links sometime back while reading on ExtJS full framework in itself is JSON.

http://www.thomasfrank.se/xml_to_json.html

http://camel.apache.org/xmljson.html

online XML to JSON converter : http://jsontoxml.utilities-online.info/

UPDATE BTW, To get JSON as added in question, HTML need to have type & content tags in it too like this or you need to use some xslt transformation to add these elements while doing JSON conversion

<?xml version="1.0" encoding="UTF-8" ?>
<type>div</type>
<content>
    <type>span</type>
    <content>Text2</content>
</content>
<content>Text2</content>

User · Answer

Thank you  Gorge Reith  Working off the solution provided by  George Reith  here is a function that furthers  1  separates out the individual  hrefs  links  because they might be useful    2  uses attributes as keys  since attributes are more descriptive   and  3  it s usable within Node js without needing Chrome by using the  jsdom  package   const jsdom   require  jsdom      npm install jsdom provides in-built Window js without needing Chrome      Function to map HTML DOM attributes to inner text and hrefs function mapDOM html string  json        treeObject              IMPT  use jsdom because of in-built Window js        DOMParser   does not provide client-side window for element access if coding in Nodejs     dom   new jsdom JSDOM html string      document   dom window document     element   document firstChild         Recursively loop through DOM elements and assign attributes to inner text object        Why attributes instead of elements  1  attributes more descriptive  2  usually important and lesser     function treeHTML element  object            var nodeList   element childNodes          if  nodeList    null               if  nodeList length                   object element nodeName           IMPT  empty    array for non-text recursivable elements  see below                 for  var i   0  i  lt  nodeList length  i                            if final text                    if  nodeList i  nodeType    3                           if  element attributes    null                               for  var j   0  j  lt  element attributes length  j                                      if  element attributes j  nodeValue         amp  amp                                       nodeList i  nodeValue                                               if  element attributes j  name      href        separate href                                         object element attributes j  name    element attributes j  nodeValue                                        else                                           object element attributes j  nodeValue    nodeList i  nodeValue                                                                                                                                                       else if non-text then recurse on recursivable elements                      else                          object element nodeName  push         if non-text push    into empty    array                        treeHTML nodeList i   object element nodeName  object element nodeName  length -1                                                                           treeHTML element  treeObject        return  json    JSON stringify treeObject    treeObject

User · Answer

Representing complex HTML documents will be difficult and full of corner cases  but I just wanted to share a couple techniques to show how to get this kind of program started  This answer differs in that it uses data abstraction and the toJSON method to recursively build the result   Below  html2json is a tiny function which takes an HTML node as input and it returns a JSON string as the result  Pay particular attention to how the code is quite flat but it s still plenty capable of building a deeply nested tree structure     all possible with virtually zero complexity   x000D   x000D     data Elem   Elem Node x000D   x000D  const Elem   e   gt     x000D    toJSON        gt     x000D      tagName   x000D        e tagName  x000D      textContent  x000D        e textContent  x000D      attributes  x000D        Array from e attributes    name  value     gt   name  value    x000D      children  x000D        Array from e children  Elem  x000D       x000D     x000D   x000D     html2json    Node - gt  JSONString x000D  const html2json   e   gt  x000D    JSON stringify Elem e   null        x000D     x000D  console log html2json document querySelector  main     x000D   lt main gt  x000D     lt h1 class  mainHeading  gt Some heading lt  h1 gt  x000D     lt ul id  menu  gt  x000D       lt li gt  lt a href   a  gt a lt  a gt  lt  li gt  x000D       lt li gt  lt a href   b  gt b lt  a gt  lt  li gt  x000D       lt li gt  lt a href   c  gt c lt  a gt  lt  li gt  x000D     lt  ul gt  x000D     lt p gt some text lt  p gt  x000D   lt  main gt  x000D   x000D   x000D    In the previous example  the textContent gets a little butchered  To remedy this  we introduce another data constructor  TextElem  We ll have to map over the childNodes  instead of children  and choose to return the correct data type based on e nodeType     this gets us a littler closer to what we might need   x000D   x000D     data Elem   Elem Node   TextElem Node x000D   x000D  const TextElem   e   gt     x000D    toJSON       gt     x000D      type  x000D         TextElem   x000D      textContent  x000D        e textContent x000D       x000D     x000D   x000D  const Elem   e   gt     x000D    toJSON        gt     x000D      type  x000D         Elem   x000D      tagName   x000D        e tagName  x000D      attributes  x000D        Array from e attributes    name  value     gt   name  value    x000D      children  x000D        Array from e childNodes  fromNode  x000D       x000D     x000D   x000D     fromNode    Node - gt  Elem x000D  const fromNode   e   gt    x000D    switch  e nodeType    x000D      case 3   return TextElem e  x000D      default  return Elem e  x000D      x000D    x000D   x000D     html2json    Node - gt  JSONString x000D  const html2json   e   gt  x000D    JSON stringify Elem e   null        x000D     x000D  console log html2json document querySelector  main     x000D   lt main gt  x000D     lt h1 class  mainHeading  gt Some heading lt  h1 gt  x000D     lt ul id  menu  gt  x000D       lt li gt  lt a href   a  gt a lt  a gt  lt  li gt  x000D       lt li gt  lt a href   b  gt b lt  a gt  lt  li gt  x000D       lt li gt  lt a href   c  gt c lt  a gt  lt  li gt  x000D     lt  ul gt  x000D     lt p gt some text lt  p gt  x000D   lt  main gt  x000D   x000D   x000D    Anyway  that s just two iterations on the problem  Of course you ll have to address corner cases where they come up  but what s nice about this approach is that it gives you a lot of flexibility to encode the HTML however you wish in JSON     and without introducing too much complexity  In my experience  you could keep iterating with this technique and achieve really good results  If this answer is interesting to anyone and would like me to expand upon anything  let me know      Related  Recursive methods using JavaScript  building your own version of JSON stringify

User · Answer

I had a similar issue where I wanted to represent HTML as JSON in the following way    For HTML text nodes  use a string For HTML elements  use an array with    The  tag  name of the element An object  mapping attribute keys to attribute values The  inlined  list of children nodes    Example    lt div gt     lt span gt text lt  span gt Text2  lt  div gt    becomes        div               span        text        Text2        I wrote a function which handles transforming a DOM Element into this kind of JS structure  You can find this function at the end of this answer  The function is written in Typescript  You can use the Typescript playground to convert it to clean JavaScript     Furthermore  if you need to parse an html string into DOM  assign to  innerHtml   let element   document createElement  div   element innerHtml   htmlString     Also  this one is common knowledge but if you need a JSON string output  use JSON stringify            A NodeDescriptor stands for either an  HTML  Element  or for a text node     export type NodeDescriptor   ElementDescriptor   string         Array representing an HTML Element  It consists of        - The  tag  name of the element    - An object  mapping attribute keys to attribute values    - The  inlined  list of children nodes     export type ElementDescriptor        string     Record lt string  string gt         NodeDescriptor      export let htmlToJs    element  Element  trim   true   ElementDescriptor   gt       let convertElement    element  Element   ElementDescriptor   gt          let attributeObject  Record lt string  string gt             for  let   name  value   of element attributes             attributeObject name    value                let childArray  NodeDescriptor              for  let node of element childNodes             let converter   htmlToJsDispatch node nodeType           if  converter                let descriptor   converter node as any              let skip   false              if  trim  amp  amp  typeof descriptor      string                    descriptor   descriptor trim                  if  descriptor         skip   true                            if   skip  childArray push descriptor                            return  element tagName toLowerCase    attributeObject     childArray           let htmlToJsDispatch            element ELEMENT NODE   convertElement         element TEXT NODE    node  Text   string   gt  node data           return convertElement element

User · Answer

I just wrote this function that does what you want  try it out let me know if it doesn t work correctly for you     Test with an element  var initElement   document getElementsByTagName  quot html quot   0   var json   mapDOM initElement  true   console log json       Test with a string  initElement    quot  lt div gt  lt span gt text lt  span gt Text2 lt  div gt  quot   json   mapDOM initElement  true   console log json    function mapDOM element  json        var treeObject                   If string convert to document Node     if  typeof element      quot string quot             if  window DOMParser                  parser   new DOMParser                  docNode   parser parseFromString element  quot text xml quot              else      Microsoft strikes again               docNode   new ActiveXObject  quot Microsoft XMLDOM quot                  docNode async   false                docNode loadXML element                       element   docNode firstChild                   Recursively loop through DOM elements and assign properties to object     function treeHTML element  object            object  quot type quot     element nodeName          var nodeList   element childNodes          if  nodeList    null                if  nodeList length                    object  quot content quot                         for  var i   0  i  lt  nodeList length  i                          if  nodeList i  nodeType    3                            object  quot content quot   push nodeList i  nodeValue                         else                           object  quot content quot   push                              treeHTML nodeList i   object  quot content quot   object  quot content quot   length -1                                                                            if  element attributes    null                if  element attributes length                    object  quot attributes quot                         for  var i   0  i  lt  element attributes length  i                          object  quot attributes quot   element attributes i  nodeName    element attributes i  nodeValue                                                      treeHTML element  treeObject            return  json    JSON stringify treeObject    treeObject     Working example  http   jsfiddle net JUSsf   Tested in Chrome  I can t guarantee full browser support - you will have to test this    It creates an object that contains the tree structure of the HTML page in the format you requested and then uses JSON stringify   which is included in most modern browsers  IE8   Firefox 3   etc   If you need to support older browsers you can include json2 js  It can take either a DOM element or a string containing valid XHTML as an argument  I believe  I m not sure whether the DOMParser   will choke in certain situations as it is set to  quot text xml quot  or whether it just doesn t provide error handling  Unfortunately  quot text html quot  has poor browser support   You can easily change the range of this function by passing a different value as element  Whatever value you pass will be the root of your JSON map

[javascript] Map HTML to JSON

Examples related to javascript

Examples related to html

Examples related to json