Scalable big data computing for the personalization of machine learned models and its application to automatic speech recognition service

We observe that the recent advances in big data computing have empowered model-based services such as speech recognition, face recognition, context-aware service, and many other services. Various sources of user's logs can be utilized in remodeling or adapting existing models to improve the quality of service. We propose a system that can support store/retrieve data and process them in a scalable manner. Recently advances in ASR and big data technologies drive more personalized services in many areas of services. A speaker adaptation is one good example which requires huge computation cost in creating a personalized acoustic model and corresponding language model over 100s millions of Samsung product users. We propose a personalized and scalable ASR system powered by the big data infrastructure which brings data-driven personalized opportunities to voice-enabled services such as voice-to-text transcriber, voice-enabled web search in a peta bytes scale. We verify the feasibility of speaker adaptation based on 107 testers' recordings and obtain about 10% of recognition accuracy. We study an optimal set of execution environments by executing jobs running either on Hadoop 1 or Hadoop 2 cluster, and move forward performance optimization strategies: workflow compaction, file compression, best file system selection among several distributed file systems. We devise a metric for the cost of personalized model creation to compare the efficiency of one cluster with the other cluster, and it provides the estimated total execution time for the given number of machines. We finally introduce our in-house object storage and data storage design, and their high performance compared to state-of-the art systems, optimized for voice-enabled services to effectively support small and large files.