Text Output

After scrubbing, you usually want a string back: HTML for rendering, or plain text for search indexes, email previews, and notifications. This page explains how Loofah's HTML documents and fragments serialize with to_s, text, and to_text, how entity escaping works, and the whitespace rules behind to_text.

These overrides apply to the four HTML classes (Loofah::HTML5::Document, Loofah::HTML5::DocumentFragment, and the HTML4 equivalents). XML documents and fragments use Nokogiri's own methods.

to_s and serialize

Receiver to_s returns
HTML fragment The HTML of the fragment's top-level nodes, with no wrapper
HTML document Nokogiri's normal document serialization (<html>, <head>, <body>; on CRuby, HTML4 documents also include a DOCTYPE)

For fragments, Loofah defines to_s (aliased as serialize) to serialize the children of the fragment's body element if it has one, or of the fragment itself otherwise. This guarantees you get back only your markup:

Loofah.html5_fragment("<style>foo</style><div>bar</div>").to_s
# => "<style>foo</style><div>bar</div>"

Notice that <style> is still there: parsing does not sanitize. Only scrub! does.

Other Nokogiri serializers such as to_html and to_xml keep working as usual; the project's own tests often use to_html on fragments.

text

text(options = {}) returns the concatenated text content, with HTML special characters escaped. It is fast but unaware of layout: block elements are glued together.

doc = Loofah.html5_fragment("<h1>Title</h1><div>Content<br>Next line</div>")
doc.text # => "TitleContentNext line"

What text includes:

  • For a fragment, the text of each top-level node (top-level comments are skipped).
  • For a document, the text of /html/body only; anything in <head> (such as <title> or <style>) is excluded. If the document has no body, the result is "".
  • Comments nested inside elements do not contribute text either.
  • The content of <style> and <script> elements in a fragment is included, because they are just elements to the fragment:
Loofah.html5_fragment("<style>foo</style><div>bar</div>").text  # => "foobar"
Loofah.html5_document("<style>foo</style><div>bar</div>").text  # => "bar"

text is aliased as inner_text and as to_str.

Note: Because to_str is defined, Ruby treats Loofah HTML documents and fragments as implicitly convertible to strings. Code like "Summary: " + fragment concatenates the escaped text, not the HTML. Call to_s explicitly when you want markup.

The encode_special_chars option

By default text escapes <, >, and & (via Nokogiri's encode_special_chars) so the result can be inserted into HTML safely:

frag = Loofah.html5_fragment("&lt;script&gt;alert('EVIL');&lt;/script&gt;")
frag.text                               # => "&lt;script&gt;alert('EVIL');&lt;/script&gt;"
frag.text(encode_special_chars: false)  # => "<script>alert('EVIL');</script>"

Pass encode_special_chars: false only when the destination is not HTML (a plain-text email, a terminal, a search index). The unescaped string can contain live markup if it is later rendered in a browser. Passing true, or omitting the option, gives the escaped form.

to_text

to_text(options = {}) produces readable plain text. It is slower than text but puts newlines where a reader expects them:

doc = Loofah.html5_fragment("<h1>Title</h1><div>Content<br>Next line</div>")
doc.to_text # => "\nTitle\n\nContent\nNext line\n"

How it works:

  1. Duplicate the document or fragment, so the original is not modified.
  2. Run the internal :newline_block_elements scrubber on the copy. Bottom-up, every block-level element is replaced by "\n" + its text + "\n" and every <br> by "\n".
  3. Call text(options) on the copy, so encode_special_chars works the same way as above.
  4. Collapse runs of three or more newlines (with any whitespace between them) into exactly two newlines, using Loofah.remove_extraneous_whitespace.

Examples from the test suite:

Loofah.html5_fragment("<div>tweedle<h1>beetle</h1>bottle<span>puddle</span>paddle<div>battle</div>muddle</div>").to_text
# => "\ntweedle\nbeetle\nbottlepuddlepaddle\nbattle\nmuddle\n"

Loofah.html5_fragment("<div>tweedle\n\n\t\n \nbeetle</div>").to_text
# => "\ntweedle\n\nbeetle\n"

Loofah.html5_fragment("hello<div>first line<br>second line</div>goodbye").to_text
# => "hello\nfirst line\nsecond line\ngoodbye"

Inline elements such as <span>, <b>, and <a> add no whitespace, so bottle<span>puddle</span> stays bottlepuddle. Link URLs are not included; only the link text is.

The result usually starts and ends with a newline. Call .strip if you need a compact string.

Which elements produce newlines

The set used by to_text is Loofah::Elements::LINEBREAKERS, the union of:

Constant Elements
STRICT_BLOCK_LEVEL_HTML4 address, blockquote, center, dir, div, dl, fieldset, form, h1–h6, hr, isindex, menu, noframes, noscript, ol, p, pre, table, ul
STRICT_BLOCK_LEVEL_HTML5 address, article, aside, blockquote, canvas, dd, div, dl, dt, fieldset, figcaption, figure, footer, form, h1–h6, header, hgroup, hr, li, main, nav, noscript, ol, output, p, pre, section, table, tfoot, ul, video
LOOSE_BLOCK_LEVEL dd, dt, frameset, li, tbody, td, tfoot, th, thead, tr
INLINE_LINE_BREAK br (becomes a single "\n")

Table cells (td, th) are in the set, so each cell lands on its own line rather than being separated by tabs.

Text from Untrusted HTML

text and to_text read text from whatever is in the tree. If you parse untrusted HTML and call to_text without scrubbing, text inside <script> or <style> elements in a fragment will appear in the output (escaped, so not executable, but visible). Scrub with :prune first when you want only the visible content:

Loofah.html5_fragment(untrusted).scrub!(:prune).to_text.strip

The Rails helper Loofah::Helpers.strip_tags is simply Loofah.html4_fragment(input).text, without pruning.

See Also