I just thought of one possible restraint on Singularity type growth: Procrastination.
As more people become immortal, at the edge of the Singularity, the amount of procrastination which occurs in society will amplify at exponential rates, leading to a major braking effect on growth of all kinds.