Generate Data For DMS (Dealership Management System)
Extract data from multiple XML files and load it to a data warehouse. The data comes from multiple dealers, and it contains vehicle sales and vehicle repair data (VW, Lexus, etc.).
- Daily XML files (source) are relatively large in size.
- The data inside the files contain multiple child-parent relationship, some of which are not properly structured.
- Loading a large historical data with over 100 GB in size.
- Using ETL component that would flatten the XML files and map it against the table schema before the load process.
- Creating optimized queries for faster transfer and load (indexes added).
- Store the in-process data at the memory location instead of a physical location before loading it to the data warehouse.
- Error handling with log registration.
- Notification alert for successful or failed run.
- Automating the ETL to run periodically.
All solved with technology.
We’ve had the privilege of helping numerous client companies in growth and impacting their respective industries, choose a Case to read about real-life scenarios of innovative software solutions and business transformations.