[Clarification on initial post: I speak of R^n and "directed line segments" as vectors.]
I "think" the answer was given by someone in another thread I started : Vectors come in equivalence classes, where the canonical representative vector from each class is the "position" vector with starting point at the origin. All other vectors in this class are translations of this vector. Now, all such representative vectors are in bijection with the set of points in the space (i.e., map a position vector to it's end point).
Now what's left to prove is that "algebra/operations with/on vectors" using the position vectors (which is what we always do) is well-defined (even though we are "viewing/considering" non-position vectors in a geometry or physics problem).