They rely on a mathematical model of the shape of the Earth. Measurements of Lattitude are made with reference to either:-

1) the ground/sea surface, or a generalised version of "flat" that we call the equipotential surface or Geoid.
2) the centre of the earth, and fixed axes that co-rotate with the earth (Geocentric or astronomic co-ordinates).

The two are not the same, by quite a significant amount (up to tens of km), and if you have a different model for the shape of the earth, the results of (1) are different.

The history of geodesy and cartography involves the evolution of models of the shape of the Earth from sphere through oblate spheroid with varying degrees of flattening (and deviations into prolate spheroids on occasion), and into modern complex local spheroids and global geoids.

It's not simple!

Longitude is a bit better behaved, for a variety of reasons.

Elevation is also a tricky one. Elevation compared to *what*? In general, we'd say "sea level", but onshore, there is no sea. One can imagine a system of sea-level canals cut through the continents to give a reference level, but you'd need to protect them from wind and atmospheric pressure variation, and correct for tides.

Again, it isn't simple. GPS co-ordinates are given in a simple spheroidal co-ordinate system that is a notably poor fit for elevation in Southeast Asia. You can be lying on the beach in Bali, with the waves lapping at your feet, and GPS will tell you that you are 40 m above sea level. In other areas, it reads low. All this is due to regional-scale gravity anomalies.

On a local scale, the gravity effect of mountain ranges can deflect the vertical sideways by a tiny fraction of a degree - enough to lead to arguments about position.

Most people are protected from the hellish complexity of Geodesy by the conventions adopted for mapping. Occasionally, when conventions clash, people get lost or drive of cliffs, or expensive objects get lost or things crash and people die. There is effort to avoid the worst outcomes, but they still happen occasionally, so don't use a US-calibrated navigation blindly for mission-critical location in Asia, for example. Get a local calibration...