This rule raises an issue when an HTML/XML parser object is created without explicitly specifying which parsing engine to use (typically the second positional parameter or a named parameter in the constructor).

In Python, this specifically refers to the BeautifulSoup object from the BeautifulSoup4 library.

Why is this an issue?

When you initialize an HTML parsing library without specifying which parser to use, the library automatically selects what it considers the "best available" parser on your system. This might seem convenient, but it creates several problems.

The parser that the library chooses depends on which parsing libraries are installed in your environment. Common parsing strategies include a language’s built-in HTML parser, faster third-party parsers with native code components, and more lenient parsers that prioritize compatibility. Different environments—such as your local machine, a colleague’s computer, a CI/CD server, or a production deployment—may have different parsers installed.

This environment-dependent behavior leads to inconsistent parsing results. Different parsers handle HTML differently, especially when dealing with malformed or ambiguous markup. For example:

• Parsers with native code components are faster but may be stricter about HTML structure
• Compatibility-focused parsers are more lenient and parse HTML like a web browser would
• Built-in parsers offer a middle ground and require no external dependencies

When parsing untrusted or poorly-formed HTML from external sources, these differences can produce completely different parse trees. Code that works perfectly in your development environment might fail or behave unexpectedly in production. A query that successfully locates an element with one parser might return nothing with another.

Beyond correctness issues, this inconsistency makes debugging difficult. When a bug appears only in certain environments, tracking down the cause becomes challenging if you don’t realize the parser is different.

For security-sensitive applications that parse untrusted HTML, relying on an unspecified parser introduces additional risk. Different parsers may interpret potentially malicious markup differently, and you lose control over this critical parsing behavior.

In Python’s BeautifulSoup library, the common parsers are html.parser (Python’s built-in), lxml (faster with native code), lxml-xml or xml (for XML documents), and html5lib (most lenient and browser-like).

What is the potential impact?

When HTML parsing libraries use different parsers across environments, your application may:

• Parse the same HTML differently in development versus production
• Experience bugs that only occur on certain machines or in specific deployment environments
• Fail to extract expected data from HTML documents, causing application logic errors
• Behave unpredictably when processing HTML from external sources

These issues reduce code reliability and make your application harder to maintain and debug.

How to fix it

Add the parser name as the second argument when creating a BeautifulSoup object. Python’s built-in html.parser is a good default choice as it requires no additional dependencies.

Code examples

Noncompliant code example

from bs4 import BeautifulSoup
from urllib.request import urlopen

webpage = urlopen('http://example.com')
soup = BeautifulSoup(webpage)  # Noncompliant
table = soup.find('table', {'class': 'data'})

Compliant solution

from bs4 import BeautifulSoup
from urllib.request import urlopen

webpage = urlopen('http://example.com')
soup = BeautifulSoup(webpage, 'html.parser')
table = soup.find('table', {'class': 'data'})

Resources

Documentation

• BeautifulSoup Documentation - Specifying the parser to use
• BeautifulSoup Documentation - Installing a parser