To correct
ulumuri somewhat: the important characteristics of the machine are:
(a) that the
frame is "
in track" - the front and rear wheels are in the same plane when riding in a straight line. Generally the better quality a bike is (and the less maltreatment it has had) the more likely this is to be the case.
Open frames lack torsional rigidity and are particularly bad for this, as they are for most things.
(b) that the
headset is correctly adjusted and in good condition. If it is too tight, the steering will tend to stick in one direction. The bars should turn freely without any resistance (other than friction between tyre and ground) but without any appreciable
play. If the bearing surfaces are pitted, the bars will tend to turn with sort of click stops; that's bad too. If the bars are stiff on your new bike, adjust the headset and get a new mechanic. I can only surmise that ulumuri's new bike had the headset overtightened.
(c) that the wheels are true (and the tyres fitted straight).
The geometry of the frame will also be a significant factor. A frame with short trail - such as a typical track or road racing machine or an MTB - will be twitchier and more likely to develop a mind of its own - than a shallower angled tourer or hybrid.
As noted, life is easier if you are moving at a reasonable pace, and on a smooth surface. You should be in a low enough gear to be able to maintain your pace without pushing hard on the pedals, since the asymmetric forces will force you into a wobble.
Possibly the most important point is a corollary to ulumuri's first - almost all these factors apply to riding a bike normally. When you ride a bike, you should not steer with your hands; you steer by shifting your weight - in other words with your bottom. The bars are just there as somewhere to put the brake levers, to enable you to spread your weight out a bit and to prevent sudden fore-and-aft weight shifts when you go over bumps. If you ride like that, then taking your hands a few centimetres off the bars momentarily - without changing your body position - should be a doddle. The rest is just a matter of practising more of the same, for longer and further from the bars.