I was wondering what exactly is the consensus in terms of declining an offer after accepting, if it is still early on in the process?

For instance, I accepted an offer last week so that I could promptly begin with the visa process, and 3 days letter received an unexpected interview from a school that I didn't think I had a shot at. The funding in my first offer is also on the lower end compared to this other school.

I have heard it's bad to accept and then decline, as that school is no longer able to extend offers furthermore to other students. Is this true? Or is that only if you accept and decline after April the 15th?

Whilst I am happy to attend my current school, if given an unrestricted choice, I would 100% go with the other school as it is a better research fit and higher funding.

Is there likely to be any lasting consequences? For the first school, I have talked extensively to the faculty, and they have been very warm and welcoming. Would this completely tatter the relationship and give me a bad name in the circle?