The simplest tape recorders are very simple indeed, and everything from a Walkman to a high-end audiophile deck embodies that fundamental simplicity.
The basic idea involves an electromagnet that applies a magnetic flux to the oxide on the tape. The oxide permanently "remembers" the flux it sees. A tape recorder's record head is a very small, circular electromagnet with a small gap in it, like this:
This electromagnet is tiny -- perhaps the size of a flattened pea. The electromagnet consists of an iron core wrapped with wire, as shown in the figure. During recording, the audio signal is sent through the coil of wire to create a magnetic field in the core. At the gap, magnetic flux forms a fringe pattern to bridge the gap (shown in red), and this flux is what magnetizes the oxide on the tape. During playback, the motion of the tape pulls a varying magnetic field across the gap. This creates a varying magnetic field in the core and therefore a signal in the coil. This signal is amplified to drive the speakers.
In a normal cassette player, there are actually two of these small electromagnets that together are about as wide as one half of the tape's width. The two heads record the two channels of a stereo program, like this:
When you turn the tape over, you align the other half of the tape with the two electromagnets.
When you look inside a tape recorder, you generally see something like this:
At the top of this picture are the two sprockets that engage the spools inside the cassette. These sprockets spin one of the spools to take up the tape during recording, playback, fast forward and reverse. Below the two sprockets are two heads. The head on the left is a bulk erase head to wipe the tape clean of signals before recording. The head in the center is the record and playback head containing the two tiny electromagnets. On the right are the capstan and the pinch roller, as seen below:
The capstan revolves at a very precise rate to pull the tape across the head at exactly the right speed. The standard speed is 1.875 inches per second (4.76 cm per second). The roller simply applies pressure so that the tape is tight against the capstan.