ED: Looking into it, the state of the art for actually-useful RAG is still about 60-100M params, so still way off from what you can fit on a floppy disk. Quantized, you're looking at maybe 25 floppy disks, 30-50MB at 4-bit compressed.
For running it comfortably, a '96-98 era Pentium MMX / Pentium II fully-in-ram with 64-128MB ram and a 1-4GB hard drive should be easy and run at a workable (go brew a cup of coffee ;) ) 1 tok/s out.
For at least being able to fit it in RAM, you're looking at the 1987-1990 era - a Mac II with the FDHD could support up to 64 MB of RAM, and a 386DX could handle 32-64MB on some motherboards. It would have cost tens of thousands of dollars and taken like half an hour per answer.
If you're really damned patient and willing to stream from disk, we can go back to 1984-1986 (IBM PC AT / XT + 40MB HDD). It could just keep the current hidden state and a tiny 640k weight buffer in RAM and stream the model layer by layer. Prefilling a tiny 300 token passage would take like 2 days, and a 40 token answer would take like 5 hours. On a 4.77 MHz 8088 PC, you're talking maybe 1-2 weeks per query.
And if you're a masochist... technically you could stream straight off of manually-swapped floppy disks! Then you could run it on a 1977-1982 Apple IIe or Commodore 64 ;) It'd take ~200x 5 1/4" disks. Of course, merely the stage of generating a 30 word answer would take around a month and six thousand floppy swaps. ;) Once every ~7 minutes.