A professional education company updates its enterprise data ingestion process using Talend RE, increasing reporting speed and expanding the understanding of its customers.
SITUATION
Our client wanted to substitute their current NetSuite data ingestion process into an enterprise database, which would allow them to ingest multiple applications services data using API calls. They sought the expertise of a data and analytics partner to help them successfully complete this project.
Reduced Data Ingestion Time by 67%
Our team designed a new way to process data from the NetSuite application using the client’s existing system. Some of the work was performed by data analysts and architects at our Mexico Delivery Center during the development process. To make the system more reliable, we used Lambda to automatically create backups for AWS RedShift. We created multiple tasks to bring in the data using APIs. Our team made improvements to the measurements used by the reporting team to make sure the data is more accurate. We also automated processes to make the data more dependable and created new measurements to show how the platform is used and how customers behave. To stay connected with the client, we attended to their quality requests from the stakeholder team’s Business Intelligence (BI) and provided them with monthly reports. Additionally, we upgraded the client’s Talend RE (EC2 instances) to the latest versions and offered support in case there were issues with processing. Talend RE is a virtual machine (VM) that contains the software that transfers data, and on each cycle the Talend service runs the data integration on this virtual machine. Our MDC team researched how to work with it and upgraded the deprecated software in use by the client by replacing the VM’s with new ones set up by a template provided by Talend.
RESULTS
Our teams have made significant improvements in processing NetSuite data, resulting in fewer failures. The client’s enterprise data now maintains automated backups for the past seven days. Additionally, we reduced the data ingestion time from one hour to 20 minutes. These advancements provide valuable insights into the company’s direction. Furthermore, they can now utilize data from external applications to generate new reports that offer valuable information about customer usage patterns.
