It's way too early to adjust sights. You need to work up a load that gives the smallest, most consistent groups. Make sure your patches look good, not tears, burn outs, etc. When you've achieved what your barrel likes, and you get that smallest, perfect group, then adjust your sights for center.
Make only one change at a time. Patch thickness, OR, ball size, OR, load. Decide which group is best, then go with that, adjust your sights accordingly.
Hope that helps.