Depends on what it is made of. Meteoric iron is unique because metal doesn’t naturally make that kind of pattern on Earth.
Certain elements and compounds are reactive in earth conditions. Metallic aluminum famously doesn’t exist naturally on Earth. If we found a hunk of it that wasn’t man made, we’d know it wasn’t on Earth however many thousands/millions of years ago, or it would have eroded away.
This also applies to certain isotopes. If something radioactive is too young, it can’t be from Earth. Earth has a well known age and all the crap that was here has been aging all this time.
There are also more obvious signs. Something from space smacking into Earth has an impact. If a meteor slamsinto a desert, we’re going to find a lot of natural glass around the impact site. Entire forests have been obliterated by meteors.