Skip to content

S2P2 does not compute acc and rmse metrics during training/evaluation #78

@NansyJourney

Description

@NansyJourney

Hello,

I am using EasyTPP to train and evaluate the S2P2 model. To verify this is not a dataset-specific issue, I tested on your provided Taxi dataset (from the EasyTPP benchmark) using the example configuration from examples/configs/experiment_config.yaml.

When I add metrics: ['acc', 'rmse'] to the S2P2 training config, the model trains successfully but no accuracy or RMSE values are reported in the logs — only loglike is shown. This behavior is consistent across both the Taxi dataset and my custom dataset.

For comparison, RMTPP, NHP, and THP all correctly report acc and rmse during training and evaluation.

My questions:

  1. Is it possible to compute acc and rmse for S2P2 using the thinning algorithm, similar to other models?
  2. If yes, what is the correct configuration or code change needed to enable this?
  3. If not, is this a known limitation of the current S2P2 implementation in EasyTPP, and are there plans to support it?

Thank you!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions