I totally agree!
A guest gave me a 4 stars on Location, because the taxi driver could not find in his GPS my street, because on Airbnb it is written as Mpakou and in his GPS it was written as Bakou. She told me to correct my address on Airbnb, but this is impossible anyway, because I cannot control Airbnb map's street names.
Another one gave 4 stars on Location, because she realized that 800 meters were a big distance for her to walk. I have mentioned that distance several times in my listing.
So, I also think that Location needs to be excluded or maybe with choice list as @Blake says.
In any way, if I mislead a guest by writing that my listing is near center and it is not, it is a matter of Location but a matter of Accurancy.