[javascript] Map HTML to JSON

I'm attempting map HTML into JSON with structure intact. Are there any libraries out there that do this or will I need to write my own? I suppose if there are no html2json libraries out there I could take an xml2json library as a start. After all, html is only a variant of xml anyway right?

UPDATE: Okay, I should probably give an example. What I'm trying to do is the following. Parse a string of html:

<div>
  <span>text</span>Text2
</div>

into a json object like so:

{
  "type" : "div",
  "content" : [
    {
      "type" : "span",
      "content" : [
        "Text2"
      ]
    },
    "Text2"
  ]
}

NOTE: In case you didn't notice the tag, I'm looking for a solution in Javascript

Thank you @Gorge Reith. Working off the solution provided by @George Reith, here is a function that furthers (1) separates out the individual 'hrefs' links (because they might be useful), (2) uses attributes as keys (since attributes are more descriptive), and (3) it's usable within Node.js without needing Chrome by using the 'jsdom' package:

const jsdom = require('jsdom') // npm install jsdom provides in-built Window.js without needing Chrome


// Function to map HTML DOM attributes to inner text and hrefs
function mapDOM(html_string, json) {
    treeObject = {}

    // IMPT: use jsdom because of in-built Window.js
    // DOMParser() does not provide client-side window for element access if coding in Nodejs
    dom = new jsdom.JSDOM(html_string)
    document = dom.window.document
    element = document.firstChild

    // Recursively loop through DOM elements and assign attributes to inner text object
    // Why attributes instead of elements? 1. attributes more descriptive, 2. usually important and lesser
    function treeHTML(element, object) {
        var nodeList = element.childNodes;
        if (nodeList != null) {
           if (nodeList.length) {
               object[element.nodeName] = []  // IMPT: empty [] array for non-text recursivable elements (see below)
               for (var i = 0; i < nodeList.length; i++) {
                   // if final text
                   if (nodeList[i].nodeType == 3) {
                       if (element.attributes != null) {
                           for (var j = 0; j < element.attributes.length; j++) {
                                if (element.attributes[j].nodeValue !== '' && 
                                    nodeList[i].nodeValue !== '') {
                                    if (element.attributes[j].name === 'href') { // separate href
                                        object[element.attributes[j].name] = element.attributes[j].nodeValue;
                                    } else {
                                        object[element.attributes[j].nodeValue] = nodeList[i].nodeValue;
                                    }

                                }
                           }
                       }
                   // else if non-text then recurse on recursivable elements
                   } else {
                       object[element.nodeName].push({}); // if non-text push {} into empty [] array
                       treeHTML(nodeList[i], object[element.nodeName][object[element.nodeName].length -1]);
                   }
               }
           }
        }
    }
    treeHTML(element, treeObject);

    return (json) ? JSON.stringify(treeObject) : treeObject;
}

I had a similar issue where I wanted to represent HTML as JSON in the following way:

• For HTML text nodes, use a string
• For HTML elements, use an array with:
  • The (tag) name of the element
  • An object, mapping attribute keys to attribute values
  • The (inlined) list of children nodes

Example:

<div>
  <span>text</span>Text2
</div>

becomes

[
   'div',
   {},
   ['span', {}, 'text'],
   'Text2'
]

I wrote a function which handles transforming a DOM Element into this kind of JS structure. You can find this function at the end of this answer. The function is written in Typescript. You can use the Typescript playground to convert it to clean JavaScript.

Furthermore, if you need to parse an html string into DOM, assign to .innerHtml:

let element = document.createElement('div')
element.innerHtml = htmlString

Also, this one is common knowledge but if you need a JSON string output, use JSON.stringify.

/**
 * A NodeDescriptor stands for either an (HTML) Element, or for a text node
 */
export type NodeDescriptor = ElementDescriptor | string

/**
 * Array representing an HTML Element. It consists of:
 *
 * - The (tag) name of the element
 * - An object, mapping attribute keys to attribute values
 * - The (inlined) list of children nodes
 */
export type ElementDescriptor = [
   string,
   Record<string, string>,
   ...NodeDescriptor[]
]

export let htmlToJs = (element: Element, trim = true): ElementDescriptor => {
   let convertElement = (element: Element): ElementDescriptor => {
      let attributeObject: Record<string, string> = {}
      for (let { name, value } of element.attributes) {
         attributeObject[name] = value
      }

      let childArray: NodeDescriptor[] = []
      for (let node of element.childNodes) {
         let converter = htmlToJsDispatch[node.nodeType]
         if (converter) {
            let descriptor = converter(node as any)
            let skip = false

            if (trim && typeof descriptor === 'string') {
               descriptor = descriptor.trim()
               if (descriptor === '') skip = true
            }

            if (!skip) childArray.push(descriptor)
         }
      }

      return [element.tagName.toLowerCase(), attributeObject, ...childArray]
   }

   let htmlToJsDispatch = {
      [element.ELEMENT_NODE]: convertElement,
      [element.TEXT_NODE]: (node: Text): string => node.data,
   }

   return convertElement(element)
}

I got few links sometime back while reading on ExtJS full framework in itself is JSON.

http://www.thomasfrank.se/xml_to_json.html

http://camel.apache.org/xmljson.html

online XML to JSON converter : http://jsontoxml.utilities-online.info/

UPDATE BTW, To get JSON as added in question, HTML need to have type & content tags in it too like this or you need to use some xslt transformation to add these elements while doing JSON conversion

<?xml version="1.0" encoding="UTF-8" ?>
<type>div</type>
<content>
    <type>span</type>
    <content>Text2</content>
</content>
<content>Text2</content>

Representing complex HTML documents will be difficult and full of corner cases, but I just wanted to share a couple techniques to show how to get this kind of program started. This answer differs in that it uses data abstraction and the toJSON method to recursively build the result

Below, html2json is a tiny function which takes an HTML node as input and it returns a JSON string as the result. Pay particular attention to how the code is quite flat but it's still plenty capable of building a deeply nested tree structure – all possible with virtually zero complexity

// data Elem = Elem Node_x000D_
_x000D_
const Elem = e => ({_x000D_
  toJSON : () => ({_x000D_
    tagName: _x000D_
      e.tagName,_x000D_
    textContent:_x000D_
      e.textContent,_x000D_
    attributes:_x000D_
      Array.from(e.attributes, ({name, value}) => [name, value]),_x000D_
    children:_x000D_
      Array.from(e.children, Elem)_x000D_
  })_x000D_
})_x000D_
_x000D_
// html2json :: Node -> JSONString_x000D_
const html2json = e =>_x000D_
  JSON.stringify(Elem(e), null, '  ')_x000D_
  _x000D_
console.log(html2json(document.querySelector('main')))

<main>_x000D_
  <h1 class="mainHeading">Some heading</h1>_x000D_
  <ul id="menu">_x000D_
    <li><a href="/a">a</a></li>_x000D_
    <li><a href="/b">b</a></li>_x000D_
    <li><a href="/c">c</a></li>_x000D_
  </ul>_x000D_
  <p>some text</p>_x000D_
</main>

In the previous example, the textContent gets a little butchered. To remedy this, we introduce another data constructor, TextElem. We'll have to map over the childNodes (instead of children) and choose to return the correct data type based on e.nodeType – this gets us a littler closer to what we might need

// data Elem = Elem Node | TextElem Node_x000D_
_x000D_
const TextElem = e => ({_x000D_
  toJSON: () => ({_x000D_
    type:_x000D_
      'TextElem',_x000D_
    textContent:_x000D_
      e.textContent_x000D_
  })_x000D_
})_x000D_
_x000D_
const Elem = e => ({_x000D_
  toJSON : () => ({_x000D_
    type:_x000D_
      'Elem',_x000D_
    tagName: _x000D_
      e.tagName,_x000D_
    attributes:_x000D_
      Array.from(e.attributes, ({name, value}) => [name, value]),_x000D_
    children:_x000D_
      Array.from(e.childNodes, fromNode)_x000D_
  })_x000D_
})_x000D_
_x000D_
// fromNode :: Node -> Elem_x000D_
const fromNode = e => {_x000D_
  switch (e.nodeType) {_x000D_
    case 3:  return TextElem(e)_x000D_
    default: return Elem(e)_x000D_
  }_x000D_
}_x000D_
_x000D_
// html2json :: Node -> JSONString_x000D_
const html2json = e =>_x000D_
  JSON.stringify(Elem(e), null, '  ')_x000D_
  _x000D_
console.log(html2json(document.querySelector('main')))

<main>_x000D_
  <h1 class="mainHeading">Some heading</h1>_x000D_
  <ul id="menu">_x000D_
    <li><a href="/a">a</a></li>_x000D_
    <li><a href="/b">b</a></li>_x000D_
    <li><a href="/c">c</a></li>_x000D_
  </ul>_x000D_
  <p>some text</p>_x000D_
</main>

Anyway, that's just two iterations on the problem. Of course you'll have to address corner cases where they come up, but what's nice about this approach is that it gives you a lot of flexibility to encode the HTML however you wish in JSON – and without introducing too much complexity

In my experience, you could keep iterating with this technique and achieve really good results. If this answer is interesting to anyone and would like me to expand upon anything, let me know ^_^

Related: Recursive methods using JavaScript: building your own version of JSON.stringify