In this study, a time-varying learning algorithm (TVLA) using particle swarm optimisation (PSO) method is presented to optimise radial basis function neural networks (RBFNNs) for identification of non-linear systems. First, support vector regression (SVR) method is adopted to determine the number of hidden layer nodes, the initial parameters of the kernel and the initial weights of RBFNNs. After initialisation, an annealing robust TVLA (ARTVLA) is then applied to train the RBFNNs. In the ARTVLA, the determination of the learning rate would be an important issue for the trade-off between stability and speed of convergence. A simple and computationally efficient optimisation method, PSO, is adopted to simultaneously find a set of promising learning rates to overcome the stagnation for searching optimal solutions in training procedure of RBFNNs. The proposed SVR-based RBFNNs with ARTVLA (SVR-ARTVLA-RBFNNs) have good performance for system identification only using few hidden layer nodes. Three examples of a non-linear system, including two benchmarks and a real data set, are illustrated to show the feasibility and superiority of the proposed SVR-ARTVLA-RBFNNs for identification of non-linear systems.