Analytics and Data Summit 2018

Adopting any new technology is daunting, in addition to peer pressure, there is also a need to size up as data grows, all without breaking the bank. The original BIWA 2015 presentation was on how any IT shop can build a commodity Hadoop cluster with Dell servers, Redhat Linux, Infiband network cards and Cloudera HDFS. Most of the coding was in Java, and OLH (Oracle Loader for Hadoop) was used to load data into an Oracle 12c in-memory table (presented at OOW15). The expertise of Computer Science faculty, staff and students was used for the full project with vendor support.The group has since enhanced the cluster with SSD (Solid State Drives) and SPARK2, and written Python Spark Machine Learning code to  analyze Gigabyte size aircraft  logs. The visualization of the data using R-Oracle has been planned for the future.