This is a metaprogram which allows you to spy on the io between an RL solver and a generative model.
