I think the atrophy you describe is normal. I had to wear the boot for about a month last year when I sprained my ankle quite badly. It took a long time to recover. Even after losing the boot, I still wore sneakers, and had to be very slow and careful for a couple of months. Ankle injuries just take a long time to heal. I was worried at first that I should have done physical therapy, but with time, it did get better, and my atrophied muscle did get back to it's normal size and strength. It doesn't cause me any pain now. If you are still supposed to be wearing the boot, I would definitely keep wearing it! I wouldn't get a smaller brace without consulting your doctor first. Trying to do too much too soon will only prolong your recovery.
I broke my kneecap about 10 years ago, and was in a stabiliser for a month. I lost so much muscle in my quad (which had been very strong as I was a dancer), and to be honest, it's still slightly weaker than my other quad, and I am less steady on that leg than the other. But it doesn't cause me any problems, and I am quite an active person.