Solusi247

Summary form only given, as follows. The complete presentation was not made available for publication as part of the conference proceedings. HGrid is a Hadoop data engineering tool made by solusi247 for Map Reduce, Spark and Storm frameworks which has been built since 2011 to help programmers building server-side data processing applications by automatically generate codes based on visually designed workflow. The biggest challenge to build this tool was to make most of the functions, modules, libraries and even workflow schemes (nearly) compatible between frameworks and to make a multipurpose single Integrated Development Environment (IDE) for Big Data availlable in the desktop and on the cloud. The development of the HGrid library starts with the collection of functions needed, how they can be implemented in each framework and how the codes can be generated. HGrid generated codes were also benchmarked against some commercially availlable tools to ensure good performance. HGrid is also designed to be visual, robust and user friendly although it still need to be improved in user experience part. It is also designed for average programmers and analysts. Data engineering libraries has been continuosly developed to ease application development with target near zero programming at the programmers side. HGrid has been used largely at the telco operators in Indonesia, in the largest banks, government institutions, hospitals and military to build complex applications such as Data Lake, Mediation Device, Media Analytics and many others.