A recently released HIRISE image shows a peak quite close to the Mars south pole at 86.7 S latitude. It's appearance is that of a recent feature:

Circular Feature Which Looks Like Subsidence and Collapse (PSP_003728_0930).
http://hirise.lpl.arizona.edu/PSP_003728_0930

A similar polar cone was seen by Mars Global Surveyor but at the north pole:

Polar Cone
MGS MOC Release No. MOC2-1519, 10 July 2006.
http://www.msss.com/mars_images/moc/2006/07/10/

Two possibilities for the cone(s) origin. It could be a volcanic rootless cone (pseudocrater) or it could be a pingo.
A pingo is a periglacial feature requiring liquid water in its melt phase for its formation. A rootless cone is a volcanic feature requiring lava/liquid water interaction for its formation.
Both of these would require geologically recent liquid water near the south pole on Mars.

Pingo
http://en.wikipedia.org/wiki/Pingo

A Pingo in the Northwest Territories, Canada.

Pseudocrater. (i.e. rootless cone)
http://en.wikipedia.org/wiki/Pseudocrater

Rootless Volcanic Cones South of the Myrdalsjokull Ice Cap Glacier.

Possible pingos or pseudocraters have been seen before on Mars though at lower latitudes:

Stratigraphical and morphological evidence for pingo genesis in the Cerberus plains.
http://arrowsmith362.asu.edu/Student_Papers/pingos.pdf

Bob Clark