Improving Generative Flow Networks with Path RegularizationGenerative Flow Networks (GFlowNets) are recently proposed models for
learning stochastic policies that generate compositional objects by sequences
of actions with the probability proportional to a given reward function. The
central problem of GFlowNets is to improve their exploration and
generalization. In this work, we propose a novel path regularization method
based on optimal transport theory that places prior constraints on the
underlying structure of the GFlowNets. The prior is designed to help the
GFlowNets better discover the latent structure of the target distribution or
enhance its ability to explore the environment in the context of active
learning. The path regularization controls the flow in GFlowNets to generate
more diverse and novel candidates via maximizing the optimal transport
distances between two forward policies or to improve the generalization via
minimizing the optimal transport distances. In addition, we derive an efficient
implementation of the regularization by finding its closed form solutions in
specific cases and a meaningful upper bound that can be used as an
approximation to minimize the regularization term. We empirically demonstrate
the advantage of our path regularization on a wide range of tasks, including
synthetic hypergrid environment modeling, discrete probabilistic modeling, and
biological sequence design.
arxiv.org