DTI fiber tractography inspires unprecedented understanding of brain neural connectivity by allowing in vivo probing of the brain white-matter microstructures. However, tractography algorithms often output hundreds of thousands of fibers and thus render the fiber analysis a challenging task. By partitioning a huge number of fibers into dozens of bundles, fiber clustering algorithms make the task of analyzing fiber pathways relatively much easier. However, most contemporary fiber clustering methods rely on fiber geometrical information only, ignoring the more important anatomical aspects of fibers. We propose in this paper a hierarchical atlas-based fiber clustering method which utilizes multi-scale fiber neuroanatomical features to guide the clustering. In particular, for each level of the hierarchical clustering, specific scaled ROIs at the atlas are first diffused along the fiber directions, with the spatial confidence of diffused ROIs gradually decreasing from 1 to 0. For each fiber, a fuzzy associativity vector is defined to keep track of the maximal spatial confidences that the fiber can have over all diffused ROIs, thus giving the anatomical signature of the fiber. Based on the associativity vectors and the ROI covariance matrix, the Mahalanobis distance between two fibers is then calculated for fiber clustering using spectral graph theory. The same procedure is iterated over coarse-to-fine ROI scales, leading to a hierarchical clustering of the fibers. Experimental results indicate that reasonable fiber clustering results can be achieved by the proposed method.