Exploring multi-granularity balance strategy for class incremental learning via three-way granular computing.

Yan Xian, Hong Yu, Ye Wang, Guoyin Wang
Author Information
  1. Yan Xian: Chongqing Key Laboratory of Computational Intelligence, Chongqing University of Posts and Telecommunications, No.2 Chongwen Road, Chongqing, 400065, China.
  2. Hong Yu: Chongqing Key Laboratory of Computational Intelligence, Chongqing University of Posts and Telecommunications, No.2 Chongwen Road, Chongqing, 400065, China. yuhong@cqupt.edu.cn.
  3. Ye Wang: Chongqing Key Laboratory of Computational Intelligence, Chongqing University of Posts and Telecommunications, No.2 Chongwen Road, Chongqing, 400065, China.
  4. Guoyin Wang: National Center for Applied Mathematics in Chongqing, Chongqing Normal University, No. 37 Middle University Road, Chongqing, 401331, China.

Abstract

Class incremental learning (CIL) is a specific scenario in incremental learning. It aims to continuously learn new classes from the data stream, which suffers from the challenge of catastrophic forgetting. Inspired by the human hippocampus, the CIL method for replaying episodic memory offers a promising solution. However, the limited buffer budget restricts the number of old class samples that can be stored, resulting in an imbalance between new and old class samples during each incremental learning stage. This imbalance adversely affects the mitigation of catastrophic forgetting. Therefore, we propose a novel CIL method based on multi-granularity balance strategy (MGBCIL), which is inspired by the three-way granular computing in human problem-solving. In order to mitigate the adverse effects of imbalances on catastrophic forgetting at fine-, medium-, and coarse-grained levels during training, MGBCIL introduces specific strategies across the batch, task, and decision stages. Specifically, a weighted cross-entropy loss function with a smoothing factor is proposed for batch processing. In the process of task updating and classification decision, contrastive learning with different anchor point settings is employed to promote local and global separation between new and old classes. Additionally, the knowledge distillation technology is used to preserve knowledge of the old classes. Experimental evaluations on CIFAR-10 and CIFAR-100 datasets show that MGBCIL outperforms other methods in most incremental settings. Specifically, when storing 3 exemplars on CIFAR-10 with Base2 Inc2 setting, the average accuracy is improved by up to 9.59% and the forgetting rate is reduced by up to 25.45%.

Keywords

References

  1. Proc Mach Learn Res. 2017;70:3987-3995 [PMID: 31909397]
  2. Nat Commun. 2020 Aug 13;11(1):4069 [PMID: 32792531]
  3. Science. 1994 Jul 29;265(5172):676-9 [PMID: 8036517]
  4. IEEE Trans Pattern Anal Mach Intell. 2018 Dec;40(12):2935-2947 [PMID: 29990101]
  5. Brain Inform. 2014 Dec;1(1-4):1-10 [PMID: 27747523]
  6. Nat Rev Neurosci. 2018 Dec;19(12):744-757 [PMID: 30356103]
  7. IEEE Trans Pattern Anal Mach Intell. 2024 Dec;46(12):9851-9873 [PMID: 39012754]
  8. IEEE Trans Pattern Anal Mach Intell. 2024 Aug;46(8):5362-5383 [PMID: 38407999]

Grants

  1. BYJS202304/Doctoral Innovation Talent Program of Chongqing University of Posts and Telecommunications
  2. 62221005/National Natural Science Foundation of China
  3. 62221005/National Natural Science Foundation of China

Word Cloud

Created with Highcharts 10.0.0learningincrementalforgettingoldCILnewclassescatastrophicclassMGBCILgranularcomputingClassspecifichumanmethodmemorysamplesimbalancemulti-granularitybalancestrategythree-waybatchtaskdecisionSpecificallysettingsknowledgeCIFAR-10scenarioaimscontinuouslylearndatastreamsufferschallengeInspiredhippocampusreplayingepisodicofferspromisingsolutionHoweverlimitedbufferbudgetrestrictsnumbercanstoredresultingstageadverselyaffectsmitigationThereforeproposenovelbasedinspiredproblem-solvingordermitigateadverseeffectsimbalancesfine-medium-coarse-grainedlevelstrainingintroducesstrategiesacrossstagesweightedcross-entropylossfunctionsmoothingfactorproposedprocessingprocessupdatingclassificationcontrastivedifferentanchorpointemployedpromotelocalglobalseparationAdditionallydistillationtechnologyusedpreserveExperimentalevaluationsCIFAR-100datasetsshowoutperformsmethods whenstoring3exemplars onwith Base2Inc2 settingaverageaccuracyimproved959%ratereduced2545%ExploringviaContrastiveEpisodicImbalanceThree-way

Similar Articles

Cited By

No available data.