I have been told elsewhere that I shouldn't submit my extension application and then leave the country even briefly before receiving the decision/new visa.
Yes, that's correct. Leaving the UK while you have a visa application pending will invalidate the application.
Will it be okay to travel to France and back at that time as long as I submit my extension application when I get back but before 13 sept 2022?
It's not ideal, but you can do that, as long as you are able to show the UK immigration officer that you will be submitting your new FLR(M) application before the end of 13th September, as they may question whether or not you will have time to apply before your visa expires.
If you can, I would have all your documents ready to apply and your online application filled out and saved before you go to France, so that all you have to do when you get back is submit the application.
Is there anything else I should be aware of?
The other thing you can't do is travel after your current visa has expired but before the new visa has been issued.
If your current visa expires while you are waiting for your new visa application to be processed, your right to live and work in the UK will be automatically extended under Section 3C Leave while the application is processing... however Section 3C Leave only remains valid as long as you remain INSIDE the UK.
If you were to leave before you have the new visa, not only will the application be invalidated, but your right to live in the UK becomes invalid as well, which means you won't be allowed back in and so you would have to return to your home country to apply for a new spousal visa and start the 5-year journey to ILR all over again.