Insurance Automation Pipeline
"Automating multi-provider insurance comparisons via AWS-hosted reporting — 12 providers, 2 form types, running non-stop."
1. The Request
I created my Malt profile in November without expecting much from it. However, I received a request in January, which was a pleasant surprise. The client was an insurance comparison company; for each of their clients, they used insurance company portals to retrieve quotes and then offer the "best one." I learned that the advisors decided which offer was the best based on many parameters, ranging from price to coverage details.
Strangely enough, the process was not yet fully automated. The advisors had to fill out forms themselves across 12 different insurance providers, covering two product lines — auto and home insurance — for a total of 24 distinct form flows. The process was tedious, repetitive, and a clear bottleneck on how many leads the team could process per day.
They wanted me to build a tool that would connect to their internal portal and fill out external provider forms based on JSON objects retrieved from their APIs — then generate quote PDFs and store them on AWS.
2. The Process
The workflow began by gathering the necessary client data from the company's internal API. Once the data was retrieved, the script needed to authenticate with the internal portal that acted as a gateway to the various insurance providers. This step presented a technical hurdle: the portal was secured by a Two-Factor Authentication (2FA) system using Google Authenticator.
To bypass this without manual intervention, I had to manage Time-based One-Time Passwords (TOTP) programmatically, incorporating new libraries to generate the codes on the fly. From there, the process was straightforward but extensive: for each of the 12 insurance portals accessible via the gateway, the bot would navigate through the forms — auto and home — filling out the many required fields and pages to generate the final offer. Finally, the system would retrieve the generated PDF and call an API to save both the results and the document.
3. Resilience: Moving to the Cloud
This was a significant step up in complexity, marking my first time hosting automation scripts in a production cloud environment rather than running them locally. Since the automation ran unattended — continuously, not just on demand — reliability was paramount. I implemented extensive try/except blocks to ensure that a slight issue or timeout with one provider wouldn't put the entire pipeline on hold.
To maintain visibility, I added robust logging to track down errors immediately without needing direct access to the console. I had to orchestrate a data flow between two distinct APIs: one to gather the lead data and another to store the results. We ingested lead data by sending a specific Lead ID to the internal API, and once processed, the script parsed the results and saved them to the database. This architecture ensured that despite the complexity of the external portals, the internal data integrity remained consistent.
4. The Result
The final solution transformed a manual, time-consuming workflow into a "set it and forget it" operation. Advisors no longer had to spend 15 to 20 minutes manually copy-pasting data into forms across a dozen portals. Instead, the system ran continuously in the background on AWS, processing requests as they came in. Due to a Non-Disclosure Agreement (NDA), I am unable to share specific code snippets or screenshots of the client's internal systems.
The client was able to handle a significantly higher volume of leads without needing to hire additional staff. The error rate dropped, and the advisors could focus on what they did best: advising clients rather than wrestling with web forms.
5. Conclusion
Looking back, the success of this project relied heavily on adopting an iterative mindset. I didn't attempt to build the entire automation suite in a single "big bang" release. I started by simply establishing a stable connection to the API. Once that data flow was secure, I tackled the login mechanism and the 2FA challenges.
I then automated just one single insurance portal — one form type — to confirm that the entire loop (data ingestion → form filling → PDF generation → storage) was working end-to-end. From that stable proof of concept, I added provider after provider, form type after form type. This iterative approach let me constantly validate with the client, ensuring that each addition matched their expectations before moving to the next.