Skip to content

Preserve filesystem prefixes embedded in paths - #1498

Open
abdulm5 wants to merge 1 commit into
databricks:mainfrom
abdulm5:agent/preserve-filesystem-prefixes
Open

Preserve filesystem prefixes embedded in paths#1498
abdulm5 wants to merge 1 commit into
databricks:mainfrom
abdulm5:agent/preserve-filesystem-prefixes

Conversation

@abdulm5

@abdulm5 abdulm5 commented Jul 27, 2026

Copy link
Copy Markdown

Fixes #854

Summary

Preserves dbfs: and file: text when it appears inside DBFS, Volume,
or local path names while continuing to normalize leading filesystem
scheme prefixes.

Why

Path wrappers currently use global string replacement to remove filesystem
prefixes. This also removes matching text from legitimate path components,
making paths such as /dbfs: unreachable.

When recursively listing /, a returned /dbfs: directory is normalized
back to / and queued again, causing an infinite traversal loop. Restricting
normalization to the beginning of the path preserves legitimate names and
prevents this aliasing.

What changed

Interface changes

None.

Behavioral changes

  • Embedded and trailing dbfs: and file: text is preserved.
  • Leading dbfs: and file: scheme prefixes retain their existing behavior.
  • Recursive listings no longer revisit the parent when a child is named
    dbfs: or file:.

Internal changes

  • Added a shared prefix-only path normalizer.
  • Updated DBFS and Volume path construction.
  • Made local file: handling prefix-only while preserving Windows file URI
    and UNC behavior.
  • Added regression coverage for exact API paths and recursive traversal.

How is this tested?

  • tests/test_dbfs_mixins.py: 30 passed.
  • Ruff lint: passed.
  • Ruff formatting check: passed.
  • git diff --check: passed.

Signed-off-by: Abdul Mohammad <abdul.mohammad9087@gmail.com>
@github-actions

Copy link
Copy Markdown

If integration tests don't run automatically, an authorized user can run them manually by following the instructions below:

Trigger:
go/deco-tests-run/sdk-py

Inputs:

  • PR number: 1498
  • Commit SHA: bdf48436bd599008b557d670e8863a65c4c02968

Checks will be approved automatically on success.

@abdulm5
abdulm5 marked this pull request as ready for review July 28, 2026 02:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[ISSUE] Files API mishandling valid dbfs paths

1 participant