1
\$\begingroup\$

I have a product that needs to have users put content in a form that potentially contains HTML and display it back to other users. I'd like to mitigate the risk as much as possible, and I can limit users to a subset of html. My primary goal is to make sure that users can't compromise each other, and that the page itself is not compromised by poorly-formatted html (e.g. omitted closing tags).

My backend is Python. Unfortunately I've been asked to not include use a third party solution.

I'm structuring my approach based on this regular expression:

(?P<content>.*?)                # Content up to next tag
(?P<markup>                     # Entire tag
  <!\[CDATA\[(?P<cdata>.+?)]]>| # <![CDATA[ ... ]]>
  <!--(?P<comment>.+?)-->|      # <!-- Comment -->
  </\s*(?P<close_tag>\w+)\s*>|  # </tag>
  <(?P<tag>\w+)                 # <tag ...
    (?P<attributes>
      (?P<attribute>\s+
# <snip>: Use this part to get the attributes out of 'attributes' group.
        (?P<attribute_name>\w+)
        (?:\s*=\s*
          (?P<attribute_value>
            [\w:/.\-]+|         # Unquoted
            (?=(?P<_v>          # Quoted
              (?P<_q>['\"]).*?(?<!\\)(?P=_q)))
            (?P=_v)
          ))?
# </snip>
      )*
    )\s*
  (?P<is_self_closing>/?)   # Self-closing indicator
  >)                        # End of tag

Here is my full code:

import re

_html_regex = r"""
        (?P<content>.*?)                # Content up to next tag
        (?P<markup>                     # Entire tag
          $|                            # End of document
          <!\[CDATA\[(?P<cdata>.+?)]]>| # <![CDATA[ ... ]]>
          <!--(?P<comment>.+?)-->|      # <!-- Comment -->
          </\s*(?P<close_tag>\w+)\s*>|  # </tag>
          <(?P<tag>\w+)                 # <tag ...
            (?P<attributes>
              (?P<attribute>\s+
        # <snip>: Use this part to get the attributes out of 'attributes' group.
                (?P<attribute_name>[\w\-]+)
                (?:\s*=\s*
                  (?P<attribute_value>
                    [\w:/.\-]+|         # Unquoted
                    (?=(?P<_v>          # Quoted
                      (?P<_q>'|\"|\\\").*?(?<!\\)(?P=_q)))
                    (?P=_v)
                  ))?
        # </snip>
              )*
            )\s*
          (?P<is_self_closing>/?)   # Self-closing indicator
          >)                        # End of tag
        """

_html_attr_regex = r"""
        (?P<attribute_name>\w+)
        (?:\s*=\s*
          (?P<attribute_value>
            [\w:/.\-]+|         # Unquoted
            (?=(?P<_v>          # Quoted
              (?P<_q>['\"]).*?(?<!\\)(?P=_q)))
            (?P=_v)
          ))?
        """

def _check_href(href):
  href = href.strip()
  if href[0] == '"' or href[0] == "'":
    href = href[1:-1].strip()

  if href.startswith('javascript:'):
    return '"#"'

  return '"' + href + '"'

_no_attributes = {}

_allowed_tags = {
  'a': {'href': _check_href},
  'b': _no_attributes,
  'blockquote': _no_attributes,
  'div': _no_attributes,
  'h1': _no_attributes,
  'h2': _no_attributes,
  'h3': _no_attributes,
  'h4': _no_attributes,
  'h5': _no_attributes,
  'h6': _no_attributes,
  'hr': _no_attributes,
  'i': _no_attributes,
  'li': _no_attributes,
  'ol': _no_attributes,
  'p': _no_attributes,
  'pre': _no_attributes,
  'table': _no_attributes,
  'tbody': _no_attributes,
  'td': _no_attributes,
  'th': _no_attributes,
  'tr': _no_attributes,
  'u': _no_attributes,
  'ul': _no_attributes,
}

_self_closing_tags = [
  'hr',
]


def cleanup_html(html):
  if not html:
    return html

  tag_stack = []
  output = ""

  for match in re.finditer(_html_regex, html, re.DOTALL | re.VERBOSE):
    content = match.group('content')
    tag = match.group('tag')
    attributes = match.group('attributes')
    close_tag = match.group('close_tag')
    is_self_closing = match.group('is_self_closing')

    if content:
      output += content

    if close_tag and close_tag in tag_stack:
      while tag_stack[-1] != close_tag:
        output += '</%s>' % tag_stack.pop()

    elif tag and tag in _allowed_tags:
      output += '<' + tag
      for attr in re.finditer(_html_attr_regex, attributes, re.DOTALL | re.VERBOSE):
        attr_name = attr.group('attribute_name')
        attr_value = attr.group('attribute_value')
        if attr_name in _allowed_tags[tag]:
          output += ' ' + attr_name
          if attr_value:
            output += '=' + _allowed_tags[tag][attr_name](attr_value)

      if is_self_closing:
        output += '/>'
      else:
        if tag not in _self_closing_tags:
          tag_stack.append(tag)
        output += '>'

  while tag_stack:
    output += '</%s>' % tag_stack.pop()

  return output

The regex and some poorly formatted sample text are pre-loaded into https://regex101.com/r/vuhnKS/1/ for playing with.

The regex should allow fairly ugly HTML, but I think it does a fairly good job of cleaning it up. There's a whitelist of allowed tags in there, and <style> and <script> aren't allowed. My guess is that the greatest weakness is in _check_href and something slipping through there.

Are there any identifiable flaws in this though? Are there any other tags that are low-risk that I should allow?

\$\endgroup\$
1
  • 5
    \$\begingroup\$ Not intended as a criticism of your approach. Just remembering the glory days of StackOverflow: stackoverflow.com/a/1732454/55857 \$\endgroup\$ – FMc Mar 9 at 20:25
0
\$\begingroup\$

It is generally thought that html parsing with regular-expressions is complex and difficult to maintain over time.

Python's standard library includes a basic html parser, html.parser, so it shouldn't be part of the 3rd-party exclusion set. You might try re-implementing your solution with it and then compare the two solutions.

Html.parser information:
https://docs.python.org/3/library/html.parser.html

\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.