Freight Data Automation

We used a system to store our Freight data, but found at times the freight data would go missing. We would have to locate the missing records and add them back into the system. We were asked if FME can automate the receiving of the data as well as storing this data within a SQL server database. We had many challenges in this workbench, most were around the testers we had to put in place to receive and check the data. In this workbench we will show how we emailed the data into FME, check for a valid subject, attachment and once we pass these tests, read the file and start to process the data. Utilizing an update detector to only update the data that is being sent to the server, we use a sql creator to compare the data to a subset of data from the table, since the table have over 500K records and we don't want to compare this to every record. Once all data has been loaded we send a HTML report back to the end user letting them know their data has been loaded and what was inserted and updated, as well as any bad records and reasons for the bad records. This was tested at the end of 2018 and we went live with this new system on Jan 2019 and have had positive feedback from end users and none of our data has gone missing.