The process is quite simple: you most likely won’t be able to do what you want to do in real time (IE using the tape machine like an external compressor), because I don’t know of a tape machine (other than a tape DELAY machine) that’s designed to record a sound and play it back, off tape, in the same moment. You’d most likely end up recording sound to tape, then when you’re done recording, rewinding the tape and playing it back into your computer and re-recording there.
So you’d need both a cassette machine and an audio interface with a good quality line level stereo output and input.
I had cassette recorders and hi-fi cassette decks when I was a kid though, and they’re not without problems. For instance, if they’re old or in bad condition, they’ll probably be quite hissy; if the tape’s not high quality and fresh, you might well get unwelcome distortion from the tape degrading; and because the motors & tension components are rarely perfect in non-profesional equipment, you might get audible pitch-wobbling. Like, I used to try to record pianos on consumer cassette decks and it always sounded crappy, because you really hear pitch modulation in a recording of a piano.
Maybe you love all of your cassette machine’s idiosyncracies, but…
Anyway, I’ve got the TB Reelbus plugin on my computer, and I love it - great for fattening up bass. Stick some drums through it, boost levels until you’re just maxing out the meters, then put something after it to tighten up the low end: very good for making a drum mix tougher, heavier…
To be honest I’d start in software - don’t see retro hardware as a silver bullet, it comes with a load of challenges.
