There's enormous turnover is teaching due to too large class sizes and general bullshit; I imagine some combination of removing bullshit and raising pay will make it easy for many people with credentials to come back.

One can also take a longer view and give better incentives to get the teaching credentials in the first place. It's not at all impossible to grow the number of teachers ethically.