Map HTML to JSON

Question

I m attempting map HTML into JSON with structure intact  Are there any libraries out there that do this or will I need to write my own  I suppose if there are no html2json libraries out there I could take an xml2json library as a start  After all  html is only a variant of xml anyway right   UPDATE  Okay  I should probably give an example  What I m trying to do is the following  Parse a string of html    lt div gt     lt span gt text lt  span gt Text2  lt  div gt    into a json object like so        type     div      content                   type     span          content               Text2                      Text2          NOTE  In case you didn t notice the tag  I m looking for a solution in Javascript

User · Answer

Representing complex HTML documents will be difficult and full of corner cases, but I just wanted to share a couple techniques to show how to get this kind of program started. This answer differs in that it uses data abstraction and the toJSON method to recursively build the result

Below, html2json is a tiny function which takes an HTML node as input and it returns a JSON string as the result. Pay particular attention to how the code is quite flat but it's still plenty capable of building a deeply nested tree structure – all possible with virtually zero complexity

_x000D_

// data Elem = Elem Node_x000D_
_x000D_
const Elem = e => ({_x000D_
  toJSON : () => ({_x000D_
    tagName: _x000D_
      e.tagName,_x000D_
    textContent:_x000D_
      e.textContent,_x000D_
    attributes:_x000D_
      Array.from(e.attributes, ({name, value}) => [name, value]),_x000D_
    children:_x000D_
      Array.from(e.children, Elem)_x000D_
  })_x000D_
})_x000D_
_x000D_
// html2json :: Node -> JSONString_x000D_
const html2json = e =>_x000D_
  JSON.stringify(Elem(e), null, '  ')_x000D_
  _x000D_
console.log(html2json(document.querySelector('main')))

_x000D_

<main>_x000D_
  <h1 class="mainHeading">Some heading</h1>_x000D_
  <ul id="menu">_x000D_
    <li><a href="/a">a</a></li>_x000D_
    <li><a href="/b">b</a></li>_x000D_
    <li><a href="/c">c</a></li>_x000D_
  </ul>_x000D_
  <p>some text</p>_x000D_
</main>

_x000D_

In the previous example, the textContent gets a little butchered. To remedy this, we introduce another data constructor, TextElem. We'll have to map over the childNodes (instead of children) and choose to return the correct data type based on e.nodeType – this gets us a littler closer to what we might need

_x000D_

// data Elem = Elem Node | TextElem Node_x000D_
_x000D_
const TextElem = e => ({_x000D_
  toJSON: () => ({_x000D_
    type:_x000D_
      'TextElem',_x000D_
    textContent:_x000D_
      e.textContent_x000D_
  })_x000D_
})_x000D_
_x000D_
const Elem = e => ({_x000D_
  toJSON : () => ({_x000D_
    type:_x000D_
      'Elem',_x000D_
    tagName: _x000D_
      e.tagName,_x000D_
    attributes:_x000D_
      Array.from(e.attributes, ({name, value}) => [name, value]),_x000D_
    children:_x000D_
      Array.from(e.childNodes, fromNode)_x000D_
  })_x000D_
})_x000D_
_x000D_
// fromNode :: Node -> Elem_x000D_
const fromNode = e => {_x000D_
  switch (e.nodeType) {_x000D_
    case 3:  return TextElem(e)_x000D_
    default: return Elem(e)_x000D_
  }_x000D_
}_x000D_
_x000D_
// html2json :: Node -> JSONString_x000D_
const html2json = e =>_x000D_
  JSON.stringify(Elem(e), null, '  ')_x000D_
  _x000D_
console.log(html2json(document.querySelector('main')))

_x000D_

<main>_x000D_
  <h1 class="mainHeading">Some heading</h1>_x000D_
  <ul id="menu">_x000D_
    <li><a href="/a">a</a></li>_x000D_
    <li><a href="/b">b</a></li>_x000D_
    <li><a href="/c">c</a></li>_x000D_
  </ul>_x000D_
  <p>some text</p>_x000D_
</main>

_x000D_

Anyway, that's just two iterations on the problem. Of course you'll have to address corner cases where they come up, but what's nice about this approach is that it gives you a lot of flexibility to encode the HTML however you wish in JSON – and without introducing too much complexity

In my experience, you could keep iterating with this technique and achieve really good results. If this answer is interesting to anyone and would like me to expand upon anything, let me know ^_^

Related: Recursive methods using JavaScript: building your own version of JSON.stringify

User · Answer

I just wrote this function that does what you want  try it out let me know if it doesn t work correctly for you     Test with an element  var initElement   document getElementsByTagName  quot html quot   0   var json   mapDOM initElement  true   console log json       Test with a string  initElement    quot  lt div gt  lt span gt text lt  span gt Text2 lt  div gt  quot   json   mapDOM initElement  true   console log json    function mapDOM element  json        var treeObject                   If string convert to document Node     if  typeof element      quot string quot             if  window DOMParser                  parser   new DOMParser                  docNode   parser parseFromString element  quot text xml quot              else      Microsoft strikes again               docNode   new ActiveXObject  quot Microsoft XMLDOM quot                  docNode async   false                docNode loadXML element                       element   docNode firstChild                   Recursively loop through DOM elements and assign properties to object     function treeHTML element  object            object  quot type quot     element nodeName          var nodeList   element childNodes          if  nodeList    null                if  nodeList length                    object  quot content quot                         for  var i   0  i  lt  nodeList length  i                          if  nodeList i  nodeType    3                            object  quot content quot   push nodeList i  nodeValue                         else                           object  quot content quot   push                              treeHTML nodeList i   object  quot content quot   object  quot content quot   length -1                                                                            if  element attributes    null                if  element attributes length                    object  quot attributes quot                         for  var i   0  i  lt  element attributes length  i                          object  quot attributes quot   element attributes i  nodeName    element attributes i  nodeValue                                                      treeHTML element  treeObject            return  json    JSON stringify treeObject    treeObject     Working example  http   jsfiddle net JUSsf   Tested in Chrome  I can t guarantee full browser support - you will have to test this    It creates an object that contains the tree structure of the HTML page in the format you requested and then uses JSON stringify   which is included in most modern browsers  IE8   Firefox 3   etc   If you need to support older browsers you can include json2 js  It can take either a DOM element or a string containing valid XHTML as an argument  I believe  I m not sure whether the DOMParser   will choke in certain situations as it is set to  quot text xml quot  or whether it just doesn t provide error handling  Unfortunately  quot text html quot  has poor browser support   You can easily change the range of this function by passing a different value as element  Whatever value you pass will be the root of your JSON map

User · Answer

Thank you  Gorge Reith  Working off the solution provided by  George Reith  here is a function that furthers  1  separates out the individual  hrefs  links  because they might be useful    2  uses attributes as keys  since attributes are more descriptive   and  3  it s usable within Node js without needing Chrome by using the  jsdom  package   const jsdom   require  jsdom      npm install jsdom provides in-built Window js without needing Chrome      Function to map HTML DOM attributes to inner text and hrefs function mapDOM html string  json        treeObject              IMPT  use jsdom because of in-built Window js        DOMParser   does not provide client-side window for element access if coding in Nodejs     dom   new jsdom JSDOM html string      document   dom window document     element   document firstChild         Recursively loop through DOM elements and assign attributes to inner text object        Why attributes instead of elements  1  attributes more descriptive  2  usually important and lesser     function treeHTML element  object            var nodeList   element childNodes          if  nodeList    null               if  nodeList length                   object element nodeName           IMPT  empty    array for non-text recursivable elements  see below                 for  var i   0  i  lt  nodeList length  i                            if final text                    if  nodeList i  nodeType    3                           if  element attributes    null                               for  var j   0  j  lt  element attributes length  j                                      if  element attributes j  nodeValue         amp  amp                                       nodeList i  nodeValue                                               if  element attributes j  name      href        separate href                                         object element attributes j  name    element attributes j  nodeValue                                        else                                           object element attributes j  nodeValue    nodeList i  nodeValue                                                                                                                                                       else if non-text then recurse on recursivable elements                      else                          object element nodeName  push         if non-text push    into empty    array                        treeHTML nodeList i   object element nodeName  object element nodeName  length -1                                                                           treeHTML element  treeObject        return  json    JSON stringify treeObject    treeObject

User · Answer

I had a similar issue where I wanted to represent HTML as JSON in the following way    For HTML text nodes  use a string For HTML elements  use an array with    The  tag  name of the element An object  mapping attribute keys to attribute values The  inlined  list of children nodes    Example    lt div gt     lt span gt text lt  span gt Text2  lt  div gt    becomes        div               span        text        Text2        I wrote a function which handles transforming a DOM Element into this kind of JS structure  You can find this function at the end of this answer  The function is written in Typescript  You can use the Typescript playground to convert it to clean JavaScript     Furthermore  if you need to parse an html string into DOM  assign to  innerHtml   let element   document createElement  div   element innerHtml   htmlString     Also  this one is common knowledge but if you need a JSON string output  use JSON stringify            A NodeDescriptor stands for either an  HTML  Element  or for a text node     export type NodeDescriptor   ElementDescriptor   string         Array representing an HTML Element  It consists of        - The  tag  name of the element    - An object  mapping attribute keys to attribute values    - The  inlined  list of children nodes     export type ElementDescriptor        string     Record lt string  string gt         NodeDescriptor      export let htmlToJs    element  Element  trim   true   ElementDescriptor   gt       let convertElement    element  Element   ElementDescriptor   gt          let attributeObject  Record lt string  string gt             for  let   name  value   of element attributes             attributeObject name    value                let childArray  NodeDescriptor              for  let node of element childNodes             let converter   htmlToJsDispatch node nodeType           if  converter                let descriptor   converter node as any              let skip   false              if  trim  amp  amp  typeof descriptor      string                    descriptor   descriptor trim                  if  descriptor         skip   true                            if   skip  childArray push descriptor                            return  element tagName toLowerCase    attributeObject     childArray           let htmlToJsDispatch            element ELEMENT NODE   convertElement         element TEXT NODE    node  Text   string   gt  node data           return convertElement element

User · Answer

I got few links sometime back while reading on ExtJS full framework in itself is JSON   http   www thomasfrank se xml to json html  http   camel apache org xmljson html  online XML to JSON converter   http   jsontoxml utilities-online info   UPDATE BTW  To get JSON as added in question  HTML need to have type  amp  content tags in it too like this or you need to use some xslt transformation to add these elements while doing JSON conversion   lt  xml version  1 0  encoding  UTF-8    gt   lt type gt div lt  type gt   lt content gt       lt type gt span lt  type gt       lt content gt Text2 lt  content gt   lt  content gt   lt content gt Text2 lt  content gt

[javascript] Map HTML to JSON

Examples related to javascript

Examples related to html

Examples related to json