This rock was sticking out of the side of one of the mounds. It was the only place we saw anything like these vertical, tube-like structures, which stretched from the ground up to probably about my shoulder.
Several people suggested that the tubes might be skolithos — tube-shaped fossils that were probably made by some kind of ancient worm creature and turn up sometimes in sandstones. While the pictures on Wikipedia don't look very similar to what I saw, there are apparently lots of different forms these things (and similar tube fossils) can take.