One thing about loading blocks, is they should be finished like a rifle, so the lube does not soak into the wood, from the patch.
I made one once, using a piece of UHMW- our version of HDPE. It is available in a number of different colours so maybe brown is also available along with black.
It is easily cut with drills to the correct size for your patch and ball. I use a narrow square file to cut 4 knotches inside each cavity, 4 points of the compass to make pushing the ball and patch into & out of the block a little easier without excessively wiping off the lube. As well, it will not absorb lube from the patch.