Model training questions about success rate and model config

[zts-hdu]

Dear SEM Author,
In the original paper, the SEM-GD model was evaluated on the single-task stack_blocks_three under the multi-task mode, achieving an 80% success rate. I replicated this task using the same configuration as described in the paper but found 0% success rate. Could there be an issue with my configuration, or is it necessary to run 1600 episodes for each task?

1. My dataset: Robotwin2.0 collected 16 tasks identical to the original text (100 episodes per task), totaling 1600 datasets (compressed by HDF5 to about 240G), and then converted them from HDF5 to LMDB format (converted to 40G)

2. My condfig and parameters:
   config = dict(     hist_steps=1,     pred_steps=64,     chunk_size=8,     embed_dims=256,     with_depth=True,     with_depth_loss=True,     min_depth=0.01,     max_depth=1.2,     num_depth=128,     batch_size=4,     **max_step=160000**,# # multi-task step=16k single-task step=8k     step_log_freq=25,     save_step_freq=10000,     num_workers=8,     **lr=1e-4,**     # checkpoint="/home/zts/RoboOrchardLab_pre_challenge_cup_final/projects/sem/robotwin_0/ckpt/groundingdino_swint_ogc_mmdet-822d7e9d-rename.pth",     checkpoint="/home/zts/SEM/RoboOrchardLab/dino-swin-pt/swin_base_patch4_window7_224_22k.pth",     bert_checkpoint="/home/zts/RoboOrchardLab_pre_challenge_cup_final/projects/sem/robotwin_0/ckpt/bert-base-uncased",     data_path="/media/zts/新加卷1/episode_1600_datasets_lmdb/lmdb",  # 1600 episodes with 16 tasks     urdf="/home/zts/RoboOrchardLab_pre_challenge_cup_final/projects/sem/RoboTwin/assets/embodiments/aloha-agilex-1/urdf/arx5_description_isaac.urdf",     **multi_task=True,**     task_names=["stack_blocks_three"],

3. Other model configurations have not been modified!

4. Graphics card: 4060ti CUDA12.1 torch2.4.1 Consistent with RoboWin

Looking forward to receiving your sincere reply！

[zts-hdu]

After experimentation, the original compressed dataset was re-collected, resulting in a dataset with uncompressed RGB fields (originally 40GB - now 44GB). It was found that the dataset quality had no impact on the success rate in multi-task mode.
I also tried single-task decoupling (training and evaluating only the stack_blocks_three task), and found that the evaluation success rate was normal under single-task conditions.
Training dataset: 1600 stack_blocks_three episodes
Training config:

Evaluation config is the same as above;
Evaluation results:

Therefore, I used a multi-task dataset for training, which is consistent with the one in the paper: 1600 multi-task data points (100 episodes each).
The training batch size is 8, the training steps are 160k, and the graphics cards used for training are Nvidia 4060ti and 5070ti. Other configurations are as follows:

The inference evaluation configuration is as follows, with multi-task changed to False and task_names changed to the single-task ["stack_blocks_three"]:

The evaluation result is 0%, as shown in the screenshot below:

My problem:
(1) Do I have any specific configuration? Aside from the changes to the config file mentioned above, there are no other changes.
(2) Based on my understanding of the paper and experiment, I reproduced the above procedure. I'm wondering if there's a misunderstanding on my part. Do any of the authors have any suggestions? Thank you very much for your enthusiastic and patient replies.

[linxuewu]

Check the training loss curve for normality and check for divergence.

[zts-hdu]

Check the training loss curve for normality and check for divergence.

Thanks a lot for your reply,I will try then.

[zts-hdu]

Check the training loss curve for normality and check for divergence.

This is my training loss curve after 26424 steps trained. Does this curve have any error?

[linxuewu]

This loss curve is ok.

Is the evaluation result of the model corresponding to this curve 0?

[zts-hdu]

This loss curve is ok.

Is the evaluation result of the model corresponding to this curve 0?

yes,and the evaluation result is also 0%.