Medical officials in Gaza said the man, who was 75 and whose body was only discovered on Saturday when it became light, had been shot several times.
They added that his family had reported he had been missing for two days.
The army spokesman said troops had spotted the man during darkness hours in an area that the military designated a combat zone that is off limits because of many attacks attempted by Palestinian militants on border patrols.
It was the latest fatal incident on the Gaza border since a Palestinian demonstrator was killed by troops on April 28.
Israel withdrew from Gaza in 2005 following 38 years of occupation. After Hamas took over Gaza in a 2007 Palestinian civil war, Israel declared it an enemy entity and has clamped down on its borders.
Israel invaded Gaza in a three-week offensive 17 months ago to force an end to rocket fire by Hamas and other groups aimed at towns in southern Israel but the border remains tense and violent incidents involving troops and militants are frequent.
