An overview of the authors' ideas about the process of completing a $p$-adically normed space in the setting of vertex operator algebras. We focus in particular on the $p$-adic Heisenberg VOA and its connections with $p$-adic modular forms.