Talend Big Data Integration For Global Sales Calculation [CASE STUDY]

Revenue generation is critical for any sales function worldwide. Thus, the sales reporting of revenues holds paramount value for any organization. It comprises difficult calculation and processing. And, further, measure the efficiency and performance of the company’s systems and processes. Automation is what our customer was seeking to process the vast amount of data, and overcome their data management challenge. Also, would help them filter many database and data formats to cut down on redundant and inaccurate data.

The primary objective of the customer was to get accurate sales reporting & revenue generation as per the sales KPI’s. This information was also used to calculate sales commission & compensation for the financial year.

The current architecture collects data from different geographies & databases, which gets processed by Talend to standardize and further develop structured data for revenue calculation. Flat files were accessed directly from the Azure Data Lake using HDFS components.

Due to multiple database and data formats, various business logics were written leading to the generation of redundant and inaccurate data.

• Analyze data in real-time
• Data redundancy
• Manual intervention for revenue calculation
• Managing multiple data formats
• Manual backup of repository and codes in the absence of GIT/SVN

Rishabh Software’s big data team architected the Talend data integration solution by integrating Talend with the current systems.

The approach was divided into several blocks to simplify functionality and combine the various outputs of the blocks. The concept of chained architecture was followed, and we developed 85+ Talend jobs to ensure the functioning of all the blocks, and below steps were followed:

1. The data was first processed for any duplicates, case-sensitive formats, and other junk data along with the information.
2. The filtered data was feed into the next block for business logic, like differentiating data based on regions and product lines.
3. The output of the above was fed into the fiscal calculation wherein the difference between the financial years of the parent company and its client was calculated.
4. The product or services were then adjusted in the Fiscal year’s revenue calculation as per the result.
5. The final data was fed into the sales and revenue computation block.
6. The input files were used to be flat files stored on ADLS, and the output was fed into databases or flat files on ADLS.

A team of 15+ big data engineers was supporting the customer including a dedicated testing and integration team to validate & check the quality of output received.

• Simplified process for revenue calculation
• Improved performance and accurate revenue and sales figures
• Less manual intervention

Sales & Revenue

Product based MNC

• Talend for Big Data Integration
• MySQL
• PowerShell Scripting
• Spark
• Azure Data Lake
• HP ALM