0:00
You're listening to a Firewall Chat, part of a miniseries of the IT'S a Tech Podcast spotlighting the talented professionals that make the mission and vision of ITS possible.
0:12
Welcome to the latest IT'S a Tech Podcast Firewall Chat.
0:17
The New York State Department of Labor is a crucial organization for the thriving business community in New York State.
0:23
Employers and workers in New York count on DOL for unemployment benefits, career development resources, workforce protections, and so many other programs and incentives.
0:35
As you can imagine, with so many important functions, communication is crucial for this agency.
0:41
On today's episode, Deepti Roy, Project Manager on the DOL Dedicated team at ITS, describes the recent launch of the DOL Virtual Agent, which has made an enormous difference in reducing call volume, getting New Yorkers the resources and information they need faster and more efficiently through the responsible use of AI.
1:02
This project is the New York State Department of Labor's brand new, cutting-edge generative AI virtual agent.
1:09
It was built specifically for the employer services unit to completely transform how state employers interact with us.
1:18
So what it does is it integrates directly with our existing interactive voice response phone system.
1:27
So when the employer calls in with common questions, the virtual agent uses Google's contact center AI and dialogue flow customer experience technology to understand their questions in natural language and provide immediate and accurate answers.
1:46
So it is basically designed to give employers a fast, responsive and satisfying self-service experience, which in turn frees up human staff to handle more complex queries.
1:58
Deepti describes why the virtual agent is essential for improving day-to-day operations at the Department of Labor.
2:05
It's a historic first for the agency.
2:08
This is a massive milestone because it is the very first generative AI virtual agent implemented for any business unit within the New York State Department of Labor.
2:18
It it represents a major leap forward in the agency-wide commitment to modernizing public services.
2:26
So before this, employers would have called on traditional call centres and because of the limited staff availability, callers often face frustrating wait times.
2:38
But this tool directly addresses that bottleneck, ensuring that employers get the real-time solutions they need, when they need it, without waiting on hold.
2:48
Deepti describes the development of this important tool.
2:52
This project was a highly collaborative effort.
2:56
It united business leads and subject matter experts from the DOL, technical leads and architects from ITS and our CISO team to ensure the safe and secure deployment of AI technology.
3:10
Crucially, we partnered with Google and there went a partner, Aegean, whose industry-leading AI expertise was instrumental in developing the core solution.
3:22
So we used agile, Scrum, Kanban methodology with fixed sprints to keep development fast and flexible.
3:30
So it obviously had different phases.
3:33
The pre-launch phase involved building the knowledge base, compiling and organizing unstructured information from various PDFs and web links.
3:43
So the AI had a reliable brain, strong cloud foundations, setting up, secure, robust architecture and infrastructure, and the most critical one was extensive testing.
3:57
We ran multiple rigorous user acceptance testing cycles, end-to-end testing, and AI bot validation to ensure the system was safe, accurate and ready for prime time.
4:09
Once the tool was ready, it was time to launch.
4:13
Deepti describes the actual rollout of the virtual agent.
4:16
It actually was on time and under budget.
4:19
We kicked off the project in August 2025 and officially went live on January 12, 2026.
4:27
The project was approved with about $1.73 million budget and we successfully delivered it approximately $25,000 under budget.
4:37
So we planned a smart, phased roll out to ensure system stability.
4:42
We didn't just flip a switch for everyone at once.
4:46
We started with a phased rollout, so routing 50% of the incoming calls to the virtual agent while closely monitoring its performance with 100% rollout after a month.
4:59
So post launch we are seeing about a 37% containment rate.
5:05
That's a huge success for us because it means over a third of the call volume is handled entirely by the virtual agent.
5:14
So that immediately takes the pressure off our staff so they can focus on really complex, high touch cases.
5:21
Deepti reflects on the impact of the virtual agent, including data analysis that clearly demonstrates how New Yorkers benefit from this tool.
5:30
Well, this project provided several key takeaways that will help guide our future initiatives.
5:38
So first being data preparation.
5:40
So when a project was initiated, one of our biggest challenge was knowledge-based preparation.
5:47
Teaching a Gen AI
5:48
tool requires gathering, cleaning and structuring data from different dozens of different PDFs and documents.
5:57
So it taught us that the success of AI is entirely dependent on the quality of the data you feed it.
6:05
Then came the navigating the unknown.
6:07
Because this was a first of its kind project for the agency, navigating UAT for a conversational AI presented a steep learning curve as we initially had no established blueprint for testing this type of system.
6:26
However, through multiple rounds of UAT, we iteratively refined and improved our testing strategy, developing new frameworks to evaluate natural conversational flow rather than just testing traditional software buttons.
6:40
And despite these challenges, with strong structured governance and weekly stakeholder trudge points, we kept the project on track and resolved issues quickly to really experience this technology.
6:53
To really experience this technology, if you are a new New York State employer, you can experience this firsthand the next time you call the New York State Department of Labor Employee Services hotline.
7:05
And for more updates on how New York State is leveraging technology to improve public services, you can visit the official New York State Department of Labor website or the New York State ITS website.
