Minchao Wang: School of Computer Engineering and Science, Shanghai University, Shanghai, P.R.China.
Wu Zhang: School of Computer Engineering and Science, Shanghai University, Shanghai, P.R.China; High Performance Computing Center, Shanghai University, Shanghai, P.R.China.
Wang Ding: School of Computer Engineering and Science, Shanghai University, Shanghai, P.R.China.
Dongbo Dai: School of Computer Engineering and Science, Shanghai University, Shanghai, P.R.China.
Huiran Zhang: School of Computer Engineering and Science, Shanghai University, Shanghai, P.R.China.
Hao Xie: College of Stomatology, Wuhan University, Wuhan, P.R.China.
Luonan Chen: School of Computer Engineering and Science, Shanghai University, Shanghai, P.R.China; Key Laboratory of Systems Biology, Shanghai Institutes for Biological Sciences, Chinese Academy of Sciences, Shanghai, P.R.China.
Yike Guo: School of Computer Engineering and Science, Shanghai University, Shanghai, P.R.China; Department of Computing, Imperial College London, London, United Kingdom.
Jiang Xie: School of Computer Engineering and Science, Shanghai University, Shanghai, P.R.China.
BACKGROUNDS: Recent explosion of biological data brings a great challenge for the traditional clustering algorithms. With increasing scale of data sets, much larger memory and longer runtime are required for the cluster identification problems. The affinity propagation algorithm outperforms many other classical clustering algorithms and is widely applied into the biological researches. However, the time and space complexity become a great bottleneck when handling the large-scale data sets. Moreover, the similarity matrix, whose constructing procedure takes long runtime, is required before running the affinity propagation algorithm, since the algorithm clusters data sets based on the similarities between data pairs. METHODS: Two types of parallel architectures are proposed in this paper to accelerate the similarity matrix constructing procedure and the affinity propagation algorithm. The memory-shared architecture is used to construct the similarity matrix, and the distributed system is taken for the affinity propagation algorithm, because of its large memory size and great computing capacity. An appropriate way of data partition and reduction is designed in our method, in order to minimize the global communication cost among processes. RESULT: A speedup of 100 is gained with 128 cores. The runtime is reduced from serval hours to a few seconds, which indicates that parallel algorithm is capable of handling large-scale data sets effectively. The parallel affinity propagation also achieves a good performance when clustering large-scale gene data (microarray) and detecting families in large protein superfamilies.
