Distances seem to be set from the text field handles, which appear to be on different sides of the page numbers in your example. Perhaps setting the text alignment or justification (part of the Text > Edit function) to center would help you equalize the distance.
Perhaps I was wrong and you will have to adjust this manually with a horizontal offset. I have not found the zero offset to look wrong, but if you feel it is, you can play with it until it looks right.
Playing with items of your score (on a copy of the file for safety, of course) is a great way to learn..