Custom FS implementation, ls returns file size correctly but du does noes

[traverseda]

I'm at my wits end on this, ls counts filesize properly but du does not.

I think it's related blocks and blksize, since it seems du calculates based on that.

class ZenohFileServer:
   #Exposed FS is a universal pathlib fs
    ...
    def handle_query(self,query):
       if method == "info":
            target = self.exposed_fs / file_path.lstrip("/")
            s = target.stat()
            # os.stat_result(st_mode=33188, st_ino=82051076, st_dev=65024, st_nlink=1, st_uid=1000, st_gid=100, st_size=380, st_atime=1771447229, st_mtime=1736599589, st_ctime=1747436362)
            data = {
                "name":    file_path,
                "size":    s.st_size,
                "type":    "directory" if target.is_dir() else "file",
                "mode":    s.st_mode,
                "uid":     s.st_uid,
                "gid":     s.st_gid,
                "ino":     s.st_ino,
                "nlink":   s.st_nlink,
                "atime":   s.st_atime,
                "mtime":   s.st_mtime,
                "ctime":   s.st_ctime,
            }
            query.reply(query.key_expr, payload=json.dumps(data).encode())
            return
            
class ZenohFileClient(AbstractFileSystem):

    def info(self, path: str, **kwargs: Any) -> dict[str, Any]:
        # Try size first — success means it's a file
        try:
            raw = self._get(path, method="info").decode()
            return json.loads(raw)
        except FileNotFoundError:
            raise
        except Exception as e:
            logger.warning(f"Error getting info for {path}: {e}")
            raise FileNotFoundError(f"Error getting info for {path}: {e}")

I've tried client side calculating a virtual blocks and blksize, to no success.

    def info(self, path: str, **kwargs: Any) -> dict[str, Any]:
        # Try size first — success means it's a file
        try:
            raw = self._get(path, method="info").decode()
            data = json.loads(raw)
            data["blocks"] = (data["size"] + 511) // 512
            data["blksize"]= 4096     
            return data
        except FileNotFoundError:
            raise
        except Exception as e:
            logger.warning(f"Error getting info for {path}: {e}")
            raise FileNotFoundError(f"Error getting info for {path}: {e}")

Any tips? Documentation on what abstractfilesystem info method should be returning?

[martindurant]

The only field that should matter is "size", which is normally 0 for directories.
du() uses find/walk, which calls ls() in the naive implementation. The entries returned by ls() should be the same as from info(), and that is what I would check first.

[traverseda]

Thanks, posting these here as notes, I'll let the thread know how it goes.

(poc-zenoh-fs) ➜  poc-zenoh-fs git:(main) ls -l ./        
total 292
drwxr-xr-x 2 traverseda users   4096 Mar  6 11:19 poc_zenoh_fs.egg-info
-rw-r--r-- 1 traverseda users 254151 Mar  6 15:10 profile.pyspy
-rw-r--r-- 1 traverseda users    354 Mar  6 11:19 pyproject.toml
-rw-r--r-- 1 traverseda users   2884 Mar  3 09:04 README.md
-rw-r--r-- 1 traverseda users    253 Feb 26 14:16 shell.nix
-rw-r--r-- 1 traverseda users    462 Mar  2 13:36 test.log
-rw-r--r-- 1 traverseda users  14436 Feb 25 09:57 uv.lock
drwxr-xr-x 3 traverseda users   4096 Mar  6 14:25 zenoh_fs

#The mounted fs
(poc-zenoh-fs) ➜  poc-zenoh-fs git:(main) ls -l /tmp/foo/
total 0
drwxr-xr-x 0 traverseda users      0 Mar  6 11:19 poc_zenoh_fs.egg-info
-rw-r--r-- 1 traverseda users 254151 Mar  6 15:10 profile.pyspy
-rw-r--r-- 1 traverseda users    354 Mar  6 11:19 pyproject.toml
-rw-r--r-- 1 traverseda users   2884 Mar  3 09:04 README.md
-rw-r--r-- 1 traverseda users    253 Feb 26 14:16 shell.nix
-rw-r--r-- 1 traverseda users    462 Mar  2 13:36 test.log
-rw-r--r-- 1 traverseda users  14436 Feb 25 09:57 uv.lock
drwxr-xr-x 0 traverseda users      0 Mar  6 14:25 zenoh_fs

du -h /tmp/foo
0       /tmp/foo

It looks like du isn't going to work if I'm not returning directory sizes, which makes sense but is odd behavior. I'll calc directory sizes and see if that helps.

[martindurant]

Oh, because info(dir) is zero? That's expected indeed. Even posix ls doesn't descend into subdirectories (nor du, unless you specify), but the size of the directory item itself is either 0 on most systems, or a small value (often one block) for an on-disk filesystem.