Sort the order of branch points to the end points of a binary skeleton image?

So I m trying to order the branch points to the end points from start to end that are extracted from a skeleton binary image. When you get the branch points and end points they are in the order of the image being scanned from bottom to top, left to right (like reading a page in a book from the bottom up). However I want to be able to link from the beginning of the skeleton to the end by connecting node to node as the crow flies. In order to do so I need the order. Any suggestions how I might?