[FEATURE]: Make Plotly Express dataframe implementation-agnostic

[ehmatthes]

[FEATURE]: Make Plotly Express dataframe implementation-agnostic

Description

For a long time, there's been questions around how tightly Plotly Express depends on pandas. For example, here's a program from the Getting Started page that's supposed to show how easy it is to make a simple plot with Plotly Express:

import plotly.express as px

fig = px.bar(x=["a", "b", "c"], y=[1, 3, 2])
fig.show()

But if you run this with only plotly[express] installed, you get an error stating that pandas is required:

Traceback (most recent call last):
  File "/.../plotly/express/_core.py", line 1210, in to_named_series
    import pandas as pd
ModuleNotFoundError: No module named 'pandas'
...
NotImplementedError: Pandas installation is required if no dataframe is provided.

I think I understand this error; Plotly doesn't want to require pandas, when there are other dataframe-based libraries that people are choosing such as Polars. However, this message appears even if you have Polars installed. I think we can make Plotly Express more agnostic toward dataframe providers by looking for an available provider, rather than exiting if pandas is not found.

Why should this feature be added?

Plotly claims to not require pandas, but it still requires pandas in places where other dataframe implementations seem to work.

Mocks/ designs

I made some small changes to plotly/express/_core.py, which allows the above example to work with Polars installed. First, add a new function to look for an existing library that supports dataframes through narwhals:

def get_df_implementation():
    """Get an installed dataframe implementation."""
    for df_implementation in ("pandas", "polars"):
        try:
            return import_module(df_implementation)
        except ImportError:
            pass

    msg = (
        "A dataframe implementation such as Pandas or Polars is required if "
        "no dataframe is provided."
    )
    raise NotImplementedError(msg)

This POC version looks for pandas first, and then Polars.

The only change in the function to_named_series() is to call get_df_implemenation() in the else block, instead of doing its own check for pandas:

def to_named_series(x, name=None, native_namespace=None):
    ...
    if isinstance(x, nw.Series):
        return x.rename(name)
    elif native_namespace is not None:
        return nw.new_series(name=name, values=x, native_namespace=native_namespace)
    else:
        df_implementation = get_df_implementation()
        return nw.new_series(name=name, values=x, native_namespace=df_implementation)

The existing function process_args_into_dataframe() needs one call to get_df_implemenation() as well:

def process_args_into_dataframe(
    args, wide_mode, var_name, value_name, is_pd_like, native_namespace
):
    ...
    length = len(df_output[next(iter(df_output))]) if len(df_output) else 0

    if native_namespace is None:
        native_namespace = get_df_implementation()

    if ranges:
        import numpy as np
        ...

With these changes, the above example runs without errors when Polars is installed. When pandas and Polars are both unavailable, the error states that one of these libraries must be installed.

Notes

I'm not tied to this particular implementation in any way, this was just a proof of concept to see if we could make Plotly more agnostic about the dataframe provider.

Is this a direction that's worth pursuing? I'm happy to work on a PR if it is, or happy to see others more versed in Plotly's internals implement it.

[ehmatthes]

Working example

Save this program as main.py:

import plotly.express as px

fig = px.bar(x=["a", "b", "c"], y=[1, 3, 2])
fig.show()

Then run:

$ uv venv .venv
$ source .venv/bin/activate
$ uv pip install numpy
$ uv pip install "git+https://github.com/ehmatthes/plotly.py@df_agnostic"
$ python main.py
...
NotImplementedError: A dataframe implementation such as Pandas or Polars is required if no dataframe is provided.
$ uv pip install polars
$ python main.py
# Chart opens in browser window.

[MarcoGorelli]

hi @ehmatthes - are you suggesting that, if the user doesn't provide a dataframe, then:

• pandas
• polars

be tried in order, depending on which the user has installed?

If so, I'd suggest not doing this, I don't the user experience of different behaviours depending on which libraries they have installed is going to be great (but, i'm not a plotly maintainer so it's not my decision)

[ehmatthes]

Hi @MarcoGorelli, yes that's what I'm suggesting. I am curious to hear from people with more experience in this area if there are reasons not to take this approach.

My current understanding is that when the user passes data that isn't structured as a dataframe to a plotting function, Plotly needs to convert that to a dataframe internally. I don't think it matters to the end user whether that internal representation comes from pandas or polars, or any other dataframe implementation. I believe what's returned in the figure object is a numpy array; I believe the dataframe is just an intermediate representation of the data.

I would guess that most users have either pandas or polars installed, but not both. I don't know how accurate that is though. So this is primarily a cascading check, where the order of what's checked probably doesn't matter too much?

If this approach is reasonable, then I think the final version of get_df_implementation() could include other libraries that allow narwhals to convert a non-dataframe collection to a dataframe.