The More you Watch the Background, the Better it Gets