@ drfazl.
The correct translation is:
{O our Rabb! I have settled of my offspring in an uncultivated valley, AT Your Forbidden Sanctuary; My Rabb, that they may establish regular Salat: so let the hearts of the people incline towards them and give provisions to them of the fruits that they may give thanks}...[14:37]
The key word here is "AT". The Arabic text says "3inda" (which means AT), not "qurba" (which means near).
This means that the valley ITSELF is the bayt (Sanctuary).
Now the QUran has used 3 descriptions for that place:
Al-Bayt'ul 2aram
Al-Bayt'ul 3ateeq
Al-Bay't ul Ma3moor
Also, the Quran states that the Kaaba IS the sanctuary itself. This means that the Kaaba is a NATURAL place, not some man-made cubic idol covered with a black cloth. Also, the term "kaab" denotes something that is standing out, high, or prominent. The heel of the foot is called "kaab", because the bone sticks out. The head of a priestly order is called "kaab", because he is the prominent figure. Hence the "Kaaba" is an elevated vale (like a bowl-shaped dell) in a mountainous region.
And yes, you are 100% right. "Uncultivated" does not mean an arid and desert-like place (like the place where the current fake Kaaba is located). The Arabic text says that the vale did not have any "ZAR3" in it, which means that no human had previously settled it or ploughed its land. "Zar3" indicates human effort to plant things. Naturally occuring trees and flora are called "NABAT", not "Zar3". Hence, Abraham settled his progeny in a desolate, wilderness place that was considered sacred to the inhabitants of the region even before Abraham had migrated to there.
And this is the very same place where Allah later spoke to Moses and told him to remove his shoes, because he was in the Purified Vale (al-wady al-muqaddas). And the mountain over-looking that valley was described as being a "Tur" (meaning: a wooden mountain - mountain covered by trees), where bramble and olive trees grew. Every single Arabic interpretation in history indicates that a "tur" = wooded mountain, whereas a barren, rocky mountain is called "jabal" (not "tur").
The sight of the current Hajj is hence 100% false. And the "Tur Sina'a" mentioned in the Quran is the proper name of a spot of land containing wooded mountains, and that has nothing to do with the current Sinai Pennisula which is part of Egypt.
Peace..