Conversation
| seed=seed) | ||
| if config.shared_train_eval_env: | ||
| self._env = alf.get_env() | ||
| self._env.reset() |
There was a problem hiding this comment.
assert async_eval = False?
There was a problem hiding this comment.
Good point. Added assertion.
There was a problem hiding this comment.
Commit not pushed to the right remote?
There was a problem hiding this comment.
ah, yes, that was what happened ... now pushed
There was a problem hiding this comment.
Happened to me also. It's hard to remember, especially now that we don't change alf that often. We can probably remove the other remote.
| if config.shared_train_eval_env: | ||
| assert not self._async, "should not use async_eval in shared_train_eval_env mode" | ||
| self._env = alf.get_env() | ||
| self._env.reset() |
There was a problem hiding this comment.
Curious, why do we call env.reset() here but not in the other branch. Maybe add a comment in the code?
| for_evaluation=True, | ||
| num_parallel_environments=num_envs, | ||
| seed=seed) | ||
| if config.shared_train_eval_env: |
There was a problem hiding this comment.
Need to set the step_type in the replay buffer just before evaluation started to StepType.LAST.
There was a problem hiding this comment.
Also need to set the next step type for training to FIRST
3 participants
Provides an option to train and eval on the same shared env, mimicking the case such as training in real with only one physical env.