If this were my cat, I would keep him inside for a full day to make sure I could observe the toilet habit.
Male cats in particular are prone to problems, and if he is NOT eliminating what goes in, he could have an intestinal blockage. You don't want to know how bad that could be.
If he is otherwise acting normally, chances are he is 'going' outside, but you need to make sure ASAP.