This study compares the performance of all parameter-setting-free and self-adaptive harmony search algorithms proposed in the previous studies, which do not ask for the user to set the algorithm parameter values. Those algorithms are parameter-setting-free harmony search, Almost-parameter-free harmony search, novel self-adaptive harmony search, self-adaptive global-based harmony search algorithm, parameter adaptive harmony search, and adaptive harmony search, each of which has a distinctively different mechanism to adaptively control the parameters over iterations. Conventional mathematical benchmark problems of various dimensions and characteristics and water distribution network design problems are used for the comparison. The best, worst, and average values of final solutions are used as performance indices. Computational results show that the performance of each algorithm has a different performance indicator depending on the characteristics of optimization problems such as search space size. Conclusions derived in this study are expected to be beneficial to future research works on the development of a new optimization algorithm with adaptive parameter control. It can be considered to improve the algorithm performance based on the problem’s characteristic in a much simpler way.