The effectiveness of surrogate functions in improving the accuracy of PSO-type algorithms in an NLP task

The present article investigates a strategy for improving the performance of PSO-type algorithms when optimizing a set of parameter values, by utilising information regarding the relative values of parameters. For this study, a specific Natural Language Processing task is revisited, related to splitting an arbitrary sentence into phrases. In solving this task here, the central idea is to determine if domain-dependent information for expected near-symmetric parameter values can be exploited to reach a better solution in a given number of iterations. If there a known task-specific relation between parameters (for instance the order of magnitude of certain parameters is likely to be similar, as they correspond to similar inputs), the research question is whether such knowledge may be exploited to initially solve a simpler problem in a lower dimensionality. Thus, a hybrid strategy is proposed, that splits the PSO evolution into two phases, (i) the first one optimizing over a reduced set of grouped parameters in order to establish the relative magnitude of each group of related parameters and (ii) the second one optimizing over the entire set of parameters to fine-tune each parameter independently. Two PSO variants are evaluated on this strategy and for each case the results are compared to baselines, to investigate the quality of the solutions achieved. Experimental results indicate that the two variants behave differently with respect to this hybrid strategy, and that one of the PSO variants benefits to generate substantially better optimization solutions.