Getting the provisional license back will take about 3 weeks but I wasn't forced to hand over my passport. If I had applied by mail, I would have had to send the passport.
I also think you were quite lucky, BEG! I went to a local DVLA office and had them stamp my photo to verify it was, in fact, me... but then they took my passport and posted it off with my application to the main Swansea office.
The good news is this: Despite the standard 3 week warning, my passport returned in the mail after just 8 days, and my provisional licence showed up after 10 days.