Images for each frame?

[vadeshah]

Hi there,

I'm currently working with the UCY dataset using your loaders, which have been very convenient. In the resulting Pandas dataframe, each row is associated with a frame_id, which (from my understanding) corresponds to the annotated i-th frame in the video. I would like to work with these individual frames, and I was wondering what might be the best way to extract them. The dataset folder have the .avi files, but the corresponding individual frames are not available.

My current idea is to use a program like VLC to go through the video and extract these individual frames every 0.4 seconds, which should be manageable, but I was wondering if there is a better way to do this. I need the frames to correctly correspond to each row; ideally, if a folder of these frames is available, I would use these. Otherwise, my method should work, but I just need to make sure that I am working with the correct frames.

Apologies if this question has been asked and answered elsewhere - if so, I haven't found it yet. If you have any suggestions or comments, they would be greatly appreciated! Thank you for your time, and I hope to hear from you soon.

[vadeshah]

A follow up question - for the UCY dataset, the frame_id starts at 10. Is this 1-indexed (i.e. the 10th frame of the video), or 0-indexed (i.e. the 11th frame of the video)? Apologies if this seems pedantic, but I would like to be as precise as possible.

[amiryanj]

Hi @vadeshah

I assume an easy way to do what you want is to use OpenCV VideoCapture.
You can find an example in ui_cv.py.
You can then save each frame using cv2.imwrite() function.
Regarding your 2nd question (both the annotations and video frame indices) are 0-indexed.

[vadeshah]

Thank you for the clarifications and advice! I will mark this as closed for now, and I will let you know if I have any other questions.

[vadeshah]

Hi there!

I'm now trying to perform the same task using the ETH dataset, but I'm running into some inconsistencies with the timestamps and the frame rate of the video. In the seq_eth scenario, according to VLC, the video has been captured at 25 fps; in the dataframe, however, each row is a 0.4-second sample, but the frame_id only increases by 6 (if it were sampled at 25 fps, this should increase by 10). Does this mean that the video is actually 15 fps, or have these frame_ids been assigned incorrectly? Any clarification would be greatly appreciated!