Foldover, quasi-periodicity, and Spin-wave instabilities in ultra-thin magnetic films