Hey there, was going through the paper and the documentation on the repository and I noticed for the openwebtext dataset, you have used a validation set on which the validation loss was plotted. I wanted to know how the validation split was created from the openwebtext dataset, and what methodology did you use to split the dataset into validation and training.
Hey there, was going through the paper and the documentation on the repository and I noticed for the openwebtext dataset, you have used a validation set on which the validation loss was plotted. I wanted to know how the validation split was created from the openwebtext dataset, and what methodology did you use to split the dataset into validation and training.