Hi can you please help me with this question?

I think you are right and it mean to "keep a distance from one's neighbor" because the poet is saying that all neighbors have this imaginary wall that separates us from one another. So even if there wasn't an actual wall separating the neighbor property you both would still understand that you could not pass beyond a certain point so there isn't really a need for an actual wall.