It appears that the the amount amount of damage you take is related to air time vs actual height. I believe that the air time is being calculated as height. So being airborne for X amount of time could be either from a vertical drop, or horizontal movement while not contacting the ground in a jump, going down stairs, stepping off a house porch, a bump in the landscape, etc.
Take a small fall while walking and you take little to no damage. Do the same small fall while running and your damage increases significantly. Once you get the perks that allow you to run faster, that increases the damage you take significantly because you get a lot more air time even though your actual altitude didn’t change.
I’m sure height is in that equation somewhere, but it seems to be a far lower determining factor than air time.