It depends what types of weather patterns you want to avoid. The summer is the rainy season, so if you want to miss that, I would avoid visiting between May and August. Spring and Fall are probably the most comfortable in terms of temperature, but you can have a great visit at any time of the year. This page has more climate data to give you a more detailed look at patterns and fluctuations: https://www.climatestotravel.com/climate/mexico/oaxaca