Andrea Montanari

Activities of Data Architecture Design and Implementation with on-premise (Cloudera, Hortonworks) and cloud-based platforms (Google). Technical Coordination and Support of the Data Engineering teams, dealing with pipelines responsible for batch and real-time Data Ingestion, integrations with industrialized Machine Learning models to meet performance and scalability requirements.

My role was to design and build data pipelines in order to load historical data into Hadoop mounted on a Azure Datalake Store, using Apache Nifi and Teradata Kylo that called main Apache Big Data services (hdfs, hive, spark). I have been involved in data ingestion and preparation using Apache Nifi, PySpark, Hive and Impala, and supported the industrialization of ML models in order to calculate predictions based on drilling systems data.

I worked on the backend of a web application with the goal of connecting and controlling an industrial machine from a web interface based on visual programming (blocks). The project involved nodeJS, proprietary nodeJS libraries, JSON handling, Realtime data retrieval using nodeJS asynchronous patterns, Angular 4 as frontend builder.

I worked on a ETL project with Informatica tools (Powercenter and Cloud) which consisted in migrating on-premise data to a cloud based CRM and I used an Oracle database as staging area where to land intermediate processed records. The client requested a POC involving Informatica Data Quality suite to operate on its existing databases.

Erasmus programme