Predict only single pointmap

[stefan-baumann]

Hi,

Thanks for this cool work and open-sourcing it directly!

Given the really high inference cost (~4min for me on 64 frames on an H200), I'm wondering whether there's a "proper" way to only get the pointmap of one frame, but across all timesteps. Effectively corresponding to dense tracking starting from one frame.

Doing all pointmaps for only one time step is straightforward, but I'm unsure about the potential interactions to keep track of when trying to do it the other way around.

Best,
Stefan

[stefan-baumann]

Hi, any thoughts on this? :)

[nawazia]

Hi,

How are you doing inference?

(~4min for me on 64 frames on an H200)

From my understanding the main bottleneck is the VGGT backbone which we need for initial pose estimation etc. I'm also having the same problem so would be interested in seeing your workaround (if any)

[stefan-baumann]

For inference, I just used the standard provided inference() method, nothing special.

The VGGT backbone is comparatively cheap, it's the heads that come after it. And no, I didn't find a workaround.