Yes, that was bit difficult and I surely wouldn't recommend it for beginners or people with less experience. The mountain itself is pretty steep as can be seen from this picture taken from the road on the way home:
However what makes it very difficult is the way the wall was constructed in this area using the natural occurring rocks - hence the name "Marble Great Wall".
It's very difficult to walk on these rocks especially on the way back down. However the view was great, and the "Marble Great Wall" continues on the next mountain - I really need to go there one day!
Behind the first watch tower there was another smaller watch tower, not like any of the other towers in this area.