Make epoch mean epoch again (in streaming language modeling)
Created by: suchenzang
We currently have the following unfortunate naming: https://github.com/facebookresearch/metaseq/blob/4288451502667dda2be71a0a1a9df5066b583ae8/metaseq/tasks/streaming_language_modeling.py#L271-L290
where our training corpus is chunked up into shards, but each shard gets referenced as an epoch.
We should fix this confusing naming to make it clear that an epoch consists of shards (before repeating / looping over the same dataset again).
Relates to https://github.com/facebookresearch/metaseq/pull/189 and https://github.com/facebookresearch/metaseq/issues/166.
Note: be careful of rng state here. On restart, we want to make sure shuffled dataset is in the same order.