It depends quite a bit where you are applying. For tier one programs, you will be competing with students with 97+ quant percentiles, great letters from well known people, near perfect GPA's, and tons of math and stat courses. So if you have all these except the letters from economists because you were not an econ undergrad, then of course you will be at a disadvantage. You can make up for it by doing research and ideally publishing somewhere, or get yourself a NSF grant, which is almost a golden ticket in of itself.
Oh yeah, also most serious applicants to tier one schools probably already have a master or two under their belts, so there's that too, or they did their undergrad at a tier one school and already know the faculty there.