Internals

This page explains how Loofah works under the hood: how it extends Nokogiri, what happens step by step when you call scrub!, and the security workarounds built into the sanitizers. Read it when you need to reason about edge cases, write a sanitizer of your own, or debug surprising output.

Source Layout

File Responsibility
lib/loofah.rb Loofah.html5_support?, the parse and scrub_* shortcut methods, aliases, remove_extraneous_whitespace
lib/loofah/concerns.rb Mixins: ScrubBehavior, TextBehavior, DocumentDecorator, HtmlDocumentBehavior, HtmlFragmentBehavior
lib/loofah/html4/*.rb, html5/document*.rb, xml/*.rb The six document and fragment subclasses
lib/loofah/scrubber.rb Loofah::Scrubber, traversal, STOP / CONTINUE, the shared html5lib_sanitize step
lib/loofah/scrubbers.rb The built-in scrubbers and Loofah::Scrubbers::MAP
lib/loofah/html5/scrub.rb Element check, attribute scrubbing, allowed_uri?, scrub_css, CDATA escaping
lib/loofah/html5/safelist.rb The safelist constants
lib/loofah/html5/libxml2_workarounds.rb Attributes affected by the libxml2 escaping issue
lib/loofah/elements.rb Block-level element sets used by to_text
lib/loofah/helpers.rb Rails-style helpers (loaded separately)

How Loofah Extends Nokogiri

The document classes mix in modules:

Class Mixins
HTML4/HTML5 Document ScrubBehavior::Node, DocumentDecorator, TextBehavior, HtmlDocumentBehavior
HTML4/HTML5 DocumentFragment TextBehavior, HtmlFragmentBehavior
XML Document ScrubBehavior::Node, DocumentDecorator
XML DocumentFragment none (it overrides parse only)

DocumentDecorator#initialize registers ScrubBehavior::Node and ScrubBehavior::NodeSet as Nokogiri decorators for that document. Nokogiri applies a document's decorators to every node and node set it hands out from that document, which is how scrub! appears on nodes, node sets, and fragments (a fragment is a node whose document is a Loofah document). A source comment explains that all of scrub!'s variants live in one method with a case on the receiver because Nokogiri decides whether to decorate based on whether the module is already included.

Fragment parsing is replaced entirely: HtmlFragmentBehavior::ClassMethods#parse(tags, encoding = nil) creates a new Loofah document of the matching flavor (so the decorators are registered), sets its encoding, and calls new(doc, tags).

Document parsing calls Nokogiri's parse and then unlinks any comment that is a direct child of the document, because those nodes "don't meet the contract that scrubbers expect of a node (e.g., it can be replaced, sibling and children nodes can be created)".

What scrub! Does

When you call doc.scrub!(:prune):

  1. Resolve the scrubber. Loofah::ScrubBehavior.resolve_scrubber looks the argument up in Loofah::Scrubbers::MAP. A symbol found there is replaced by a new instance of the mapped class. If the result is not a Loofah::Scrubber, it raises Loofah::ScrubberNotFound.
  2. Choose the starting points according to the receiver: a document's root element, each child of a fragment, or the node itself. A NodeSet calls scrub! on each node.
  3. Traverse. Scrubber#traverse walks the subtree top-down or bottom-up, calling the block or scrub(node) for every node. In top-down mode a STOP return value prunes the walk.
  4. Return the receiver.

The Shared Sanitization Step

:strip, :prune, and :escape all begin with the private method Scrubber#html5lib_sanitize(node):

case node.type
when ELEMENT_NODE
  if HTML5::Scrub.allowed_element?(node.name)
    HTML5::Scrub.scrub_attributes(node)
    return CONTINUE
  end
when TEXT_NODE, CDATA_SECTION_NODE
  if HTML5::Scrub.cdata_needs_escaping?(node)
    node.before(HTML5::Scrub.cdata_escape(node))
    return STOP
  end
  return CONTINUE
end
STOP

CONTINUE means "this node is fine". STOP means "this node must be handled", and each scrubber then applies its own strategy: :strip moves the children before the node and removes it, :prune removes it, and :escape inserts a text node containing node.to_s after it and removes it. :whitewash has its own similar logic that also deletes all attributes and rejects namespaced elements.

Attribute scrubbing

Loofah::HTML5::Scrub.scrub_attributes(node) processes each attribute in order:

  1. Build the full name, including a namespace prefix (xlink:href).
  2. data-* names: keep and move on.
  3. Not in ALLOWED_ATTRIBUTES: remove.
  4. In ATTR_VAL_IS_URI: remove if allowed_uri? is false.
  5. In SVG_ATTR_VAL_ALLOWS_REF: keep only local url(#...) references and plain tokens.
  6. href / xlink:href on an element in SVG_ALLOW_LOCAL_HREF: remove unless the value starts with #.

Then it scrubs the style attribute with scrub_css, removes every non-data-* attribute whose value is empty or whitespace-only, and finally applies the libxml2 escaping workaround described below.

CDATA and Raw-Text Escaping

With Nokogiri's HTML4 parser on CRuby, the content of elements like <script> and <style> becomes CDATA nodes. An HTML5 parser in the browser may treat the same bytes differently, which led to several vulnerabilities where content that looked inert to the sanitizer became live markup in the browser (the repository's decision record lists them).

Loofah's answer is to escape <, >, and & in every CDATA node that a sanitizer visits:

  • cdata_needs_escaping?(node) is true for CDATA nodes, and, on JRuby, for text nodes whose parent is <style> (Nokogiri's JRuby HTML4 parser does not flag those as CDATA but treats them the same way).
  • cdata_escape(node) creates a new CDATA (or text) node with the escaped content, inserted before the original. The original is then removed by the scrubber's strategy.

The decision record accepts one trade-off: special characters inside an otherwise-valid <style> element are escaped too. In practice <style> is not on the safelist, so this mostly affects the text left behind by :strip.

XML documents get the same treatment with :whitewash:

xml = Loofah.xml_fragment("<p>hello<![CDATA[</p><script>alert(1)</script><p>]]>world</p>")
xml.scrub!(:whitewash).to_s
# => "<p>hello<![CDATA[&lt;/p&gt;&lt;script&gt;alert(1)&lt;/script&gt;&lt;p&gt;]]>world</p>"

The escaping is implemented by Loofah::HTML5::Scrub.escape_tags, a modified copy of Ruby 3.1's CGI.escapeHTML that escapes only <, >, and & and handles ASCII-incompatible and dummy encodings.

The libxml2 Attribute-Escaping Workaround

libxml2 2.9.2 and later deliberately leave some attribute values unescaped when serializing HTML, so that server-side include comments in URLs keep working. An attacker can exploit this: a value containing " and <!-- can break out of the attribute when the sanitized HTML is re-parsed (CVE-2018-8048).

When Nokogiri uses libxml2 (that is, on CRuby), force_correct_attribute_escaping! runs at the end of scrub_attributes. For the attributes in LibxmlWorkarounds::BROKEN_ESCAPING_ATTRIBUTES (href, action, src, and name, the last only on <a> elements), it percent-encodes spaces as %20 and double quotes as %22, mimicking libxml2's pre-2.9.2 behavior. The rest of the value is left alone.

As a result, a sanitized <a href="foo bar"> serializes as <a href="foo%20bar"> on CRuby.

Element Sets and Case

Loofah::MetaHelpers.add_downcased_set_members_to_all_set_constants runs on Loofah::Elements and Loofah::HTML5::SafeList at load time. It adds a lower-case copy of every member to every Set constant, so membership checks succeed whether a parser reports clipPath or clippath.

HTML5 Feature Detection

Loofah.html5_support? is computed once and memoized. It is true only if Gem::Version.new(Nokogiri::VERSION) > Gem::Version.new("1.14.0") and Nokogiri.uses_gumbo?. A source comment says HTML5 support needs Nokogiri >= 1.14.0 because of a subclassing fix in that release, but the comparison in the code is strict, so Nokogiri 1.14.0 itself is treated as unsupported. When it is false, lib/loofah.rb skips loading the HTML5 document classes and defines the html5_* methods to raise NotImplementedError.

See Also