Instead of refoaming, I thought why not just try with the original surround before throwing it away? So I removed the surround from the spider, used some hard photo paper to shim and reglued the surround to the spider frame. It was a lot better after that but still not 100% OK. I then found a spot where I just had to pull a little bit to get the sound good, so while playing some bass-heavy tracks I removed the surround again at that spot and then again glued it while listening if the sound was still OK. I kept pressure on it for some minutes while playing music so I could hear if it had moved a bit. Now it's dried and my speaker once again is sounding great! I could only hear a very slight bit of noise a couple of minutes ago on low volume. I'll see if it holds up, if not I know where the problem is, and can try to fix it better.
Thank you all for your advice and because I know you all love pics, here is one while shimming it!
