The question is about thousands of years ago, which is actually very young geologically speaking. The key proxies of interest for these timescales are ice records and tree rings (although others such as corals exist).
Ice cores can be used to reconstruct temperature because the ratio between the two main oxygen isotopes (atoms that are chemically identical but have slightly different masses) depends on a number of factors, including sea surface temperature. If you can control for non-temperature related factors and then use a calibration curve (which we can use known instrumental records over the past few centuries and lab experiments for), then we can use these oxygen isotope ratios to infer temperature.
Tree rings can also be used to reconstruct temperature because the growth of a tree depends on several parameters such as temperature and rainfall in a known fashion. If you know how to calibrate growth patterns for a specific tree species (which you can do by comparing tree ring patterns to known, modern temperature records) then you can in turn infer temperature changes in the past.