I also have a larger hand drill that is literally a small chuck attached to a steel shaft that has then been driven into an old file handle. This is very useful in the toolbox, but far too heavy to use with small drills.
The 0.3-4mm JT0 drill chuck. I had to look up Jacobs JT0 taper to look at the specification. I bought this particular chuck from Chronos for £29, but you can get cheaper versions from China on ebay for around £4 including postage.
My solution to this was to fit the key into the knurled cap that is threaded 14x1mm that they screws into the handle.
The handle and shaft are one piece mild steel with the JT0 taper at one end and the wide hollowed out body at the other.
The ebony and rosewood were turned and bore to fit over the shaft and main body of the handle.
I then glued the ebony and rosewood pieces of wood to the steel body with epoxy, with a small amount of titebond between the ebony and rosewood – once set I was able to hold this in the lathe by the shaft and machine, sand and polish the handle.
The result is a small, lightweight rather nice pinvice that is a delight to use. The ebony and rosewood will get better with use and age as the oil from your hand during use will add a depth of polish.
Purchased from Lee Valley in Canada. It’s their Hollow-Arbor Chuck Set. It comes with a hollow and solid arbor. I heated the last 12 mm of each arbor to soften the steel and threaded both ends 3/8″-24 tpi as it now fits a handle or a small hand drill that I have. Also useable on my lathe.