That happened to us; I just called the agency and let them know that the person no longer lived at the address. They asked lots of probing questions that all got back the answer "I don't know", but we never heard anything from them again.
One thing that they did seem particularly interested in was how I knew about the notice, since it was in an envelope addressed to the former tenant. I'm not sure if there was an issue with mail tampering from me opening someone else's post (or more likely, they suspected that I was the addressee and was trying to put them off the trail). I never admitted to opening the envelope, just repeated a few times that I had been "made aware" of the notice, which they grudgingly accepted.
If you want to be thorough, send them something registered delivery saying all these things, and keep a copy for yourself. Don't worry though, they don't care about you, just about the other guy who didn't pay his bills.
