Neural Feature Learning for Engineering Problems

Using deep neural networks as elements of engineering solutions can potentially enhance the overall performance of the system. However, most existing practices that use DNNs as black boxes make integrating DNN modules in engineering systems hard. In this paper, we address one of such difficulties: in engineering solutions, we often look for parameterized solutions that perform well in a collection of scenarios. In most problems, this means the DNN modules are trained and used in different environments. Instead of using a transfer learning or multi-task learning formulation, which are common in the literature, we argue that such problems are intrinsically about the multivariate dependence between the input, the target, and the environment parameters. Using an example of symbol detection over wireless fading channels with interference, we demonstrate that such problems can generally be formulated as a decomposition of multivariate dependence. We establish a geometric structure for the space of feature functions, based on which we develop new metrics to measure the information contents of features. We also develop some basic neural network architectures to perform geometric operations on features. With these building blocks, we discuss the steps to build a receiver that does not require any online training but can adapt to different fading scenarios when given the channel state information (CSI). We also discuss some key issues and steps to include DNN modules in general engineering systems.