I wouldn't consider 20 MPH on six inch wheels is a low speed wobble. But I was referring to the mud in what little is left of the treads causing an imbalance, along with the fact that there is no tread/traction on the road. When you installed the front end, where you didn't establish rake angle so you could get trail. Does it have trail? Can you flip the lower legs 180 degrees, then remount? If so, that would quickly answer that question as you got the axle more aft.
I tried to edit your photo with lines and words, but maybe you can see enough to see what I mean about trail. Think of a shopping cart, and how it needs to run with the mass pulling the front end. Two wheeled machines need this same concept. Mass pulling from in front of the axle, not pushing from behind it.