Context-aware task allocation for fast parallel big data processing in optical-wireless networks

MapReduce architecture has been considered as one of the most promising candidates for efficient and reliable big data mining. While current MapReduce is basically designed for data center and enterprise networks, in which a number of servers are interconnected with optical fiber cables, prospective MapReduce would be applied in optical-wireless environment such as optical-wireless data center network, fiber-wireless (FiWi) access network, and so forth. To modify MapReduce for opticalwireless hybrid network, we need to answer the fundamental research problem, “How does MapReduce architecture use optical and wireless resources for task allocation?” To answer this question, this paper reveals some challenging issues and proposes a context-aware task allocation scheme that is designed by considering characteristics of both optical and wireless communications. Our proposed task allocation scheme can minimize the completion time of big data processing. Numerical results are presented to demonstrate the effectiveness of our proposed method compared with existing task allocation schemes.