Performance can indeed deteriorate with a LOT of time but we’re usually talking well over a decade and/or heavily abused speakers and/or storage in extremely harsh environments. But as far as a new speaker is concerned, nope, no difference. Any properly designed woofer is going to have exactly the same frequency response from date of manufacturing to a reasonable life expectancy under reasonable operating conditions.
But performance most certainly won’t get better with a properly designed speaker since the engineers would have engineered the speaker with a specific frequency response in mind and they’ll need it to perform consistently if they want to deliver their intended design to customers with consistency.
Plus as mentioned already speakers are often measured as part of the whole design to manufacturing to quality assurance process (at least you’d hope so). No good doing all those measurements if all it takes is a (few?) hundred hour(s) to change the operating characteristics of the speaker, that’s just way silly. Speaker break-in is just sales crappery to try and get customers to get used to crap speakers…
…and hopefully by the time the customer realises they’ve been handed over a turd the return window is well over :D