Your main task will be to weave a credible story about why you want to become an expert in political science. This may be hard to do with just one letter. If your one recommender is good and vouches that this is the case, and you come up with a convincing
SOP for why you want to study political science now instead of just math or econ, then you will be ok. One problem many students have when applying is that they have no idea what real political science is (i.e. they say in their
SOP the reason they want to go study poli sci is because they always liked watching politics on the news). You need to make sure to get across to the ad com that this is not the case for you, even though you've taken very limited coursework. Finally, just FYI, I know for a fact my economics
LOR had very little to do with my admission to political science programs, even though this letter would have carried a good deal of weight in econ admissions. My poli sci ones on the other hand were EXTREMELY integral in this respect. I therefore suspect that your math or econ
LORs most likely won't have too much pull in getting into any given poli sci program. Afterall, 99% of the applicants are going to have a recommender that says X student is great for Y program - the difference boils down to whether the ad comm knows the recommender personally, or at the very least, their published work.