Continue pretraining on my dataset

[Keramatfar]

Thanks for your incredible models,
I want to fine tune the model using mlm on my dataset. Can i use the regular fine tuning using transformers package or i must use the current repo pretraining method?

[NohTow]

If it's for a big pre-training, you should probably go with this repo
Here's a nice starting point