Evidence suggests that the Famen Temple was built during the reign of Han Emperor Ling (156 – 189 AD) and was expanded or remodeled during the Northern Zhou Dynasty (557 to 581 AD). It was built to house early Buddhists who traveled from the West and to serve as a translation and study center. During the later part of the Northern Zhou Dynasty Buddhism was suppressed and Buddhists persecuted. The temple was destroyed and fell into disrepair.
Under the Sui and then the Tang Dynasties, Buddhism was again in favor at court. Even during the periods where Daoism received the primarily patronage of the court, the mass suppression of the Northern Zhou was not repeated.
It was renamed the Famen Temple in 618 AD and the grounds were rebuilt with the major temple buildings by Tang Emperor Wudi. It was renamed and renamed several times, but finally returned to the Famen name given at the start of the Tang Dynasty. Famen was the source of relics of Buddha for many Tang emperors. Each time the emperor wanted to seek favor with the heavens they sought support from the relics and donated generously to the temple.
Due to its history, the monks built an underground palace to house their treasures and keep them from prying eyes and hands. Over the centuries the idea of such a palace existing within the temple grounds turned into myth. They were said to have hidden the treasures of the monastery there to guard them against war, invasion, and the vagaries of suppressive political movements, but since no one could find it, the story was discounted.
The pagoda was built and burnt several times. The Tang pagoda was built of wood. During the Ming Dynasty an earthquake destroyed much of the temple and the pagoda. During the reign of Ming Emperor Wanli in 1579 a brick pagoda was built to mimic the design of the original wooden structure. The temple and pagoda were refurbished several times during the Qing Dynasty. The early Republican Army used the temple as a camp site, but in 1940 funded a restoration project to refurbish the temple and bring employment to the area. The Buddhist monks returned to the temple.