One of the main factors that turns a normal place into a desert is an absence of water. This prevents plants from growing, and in turn removes the normal process of plants capturing carbon dioxide from the air and converting it into carbon compounds.
Normally those organic compounds plus tiny bits of rock plus water form a sort of glue that keeps the ground together as a form of soil, and then plant roots can sink into it and help keep it into place even better. But no plants or water in a desert means nothing to keep the bits of rock stuck down, and so the next ingredient, wind, takes effect.
Wind picks up the bits of rock and blows them around, and the lightest forms of grit become blown away completely, perhaps out to sea, while the larger, sandy-sized bits more or less stay. So you end up with fairly uniform and often very stable dunes of sand.
And the final piece is rock erosion. Hot sun, cold nights, constant wind abrading away rock faces with blown grit. Over thousands of years it adds up into more crumbling dry rock that eventually forms more sand.