Episode 13: AI in New York State

Over the past few years, generative artificial intelligence, or AI, has been incorporated into a variety of programs and tools, inspiring some to ask what tasks could be streamlined and improved through the use of this technology.

In 2024, Governor Hochul included AI in her State of the State address, announcing the Empire AI initiative and a new policy, authored by ITS, to govern the safe and responsible use of AI to benefit all New Yorkers.

Over the past few years, as this technology has evolved, ITS has been working behind the scenes, preparing to implement AI where it makes sense, streamlining processes, making tasks easier for the talented New York State workforce and making government services more accessible for the people who need them.

On today’s episode, New York State’s new Chief Artificial Intelligence Officer, Dr. Eleonore Fournier-Tombs, will give our listeners a general overview of what is in store for the future of New York State as the scope and capabilities of AI continue to evolve.

Thank you for listening to the IT’S a Tech Podcast. For more information about ITS, visit our website at its.ny.gov. Follow us on X, LinkedIn, Instagram and Facebook.

Transcript

0:00
 You're listening to the IT's a Tech Podcast, an engaging conversation about the game-changing technology solutions being advanced by the State's Office of Information Technology Services.

 0:10
 Learn how ITS makes IT happen for more than 50 state agencies and over 20 million New Yorkers.

 0:19
 Over the past few years, generative artificial intelligence, or AI, has been incorporated into a variety of programs and tools, inspiring some to ask what tasks could be streamlined and improved through the use of this technology.

 0:34
 In 2024, Governor Hochul included AI in her State of the State address, announcing the Empire AI Initiative and a new policy authored by ITS to govern the safe and responsible use of AI to benefit all New Yorkers.

 0:49
 Over the past few years, as this technology has evolved, ITS has been working behind the scenes, preparing to implement AI where it makes sense, streamlining processes, making tasks easier for the talented New York State workforce, and making government services more accessible for the people who need them.

 1:07
 On today's episode, New York State's new Chief Artificial Intelligence officer, Doctor Eleanore Fournier-Tombs, will give our listeners a general overview of what is in store for the future of New York State as the scope and capabilities of AI continue to evolve.

 1:24
 Eleanore, first, thank you for being here with us today.

 1:27
 And second, welcome to New York State government.

 1:31
 You've been in the role of Chief AI Officer for about a month, though you've been working in this space for many years.

 1:38
 Can you talk a bit about your career path to this point, including how you became interested in artificial intelligence?

 1:45
 Well, thank you so much for having me.

 1:47
 I'm really thrilled to be here today and in ITS, which has been an amazing six weeks already.

 1:53
 Six weeks, My goodness.

 1:56
 I have done a lot of my career in data science.

 1:59
 I discovered machine learning and predictive analytics at some point in my career, and I really enjoyed that kind of predictive work and that capacity that machine learning gave.

 2:11
 And so I worked at the UN specifically on predictions related to humanitarian crises.

 2:19
 So how would you be able to figure out if there's a crisis, who's most in need, what the beneficiaries need?

 2:26
 How could you best distribute aid?

 2:28
 So a context that's important in an international context, but also at the state level, it translates very well to kind of early warning systems that we might have here and crisis response.

 2:40
 And at a certain point in, I would say the, you know, 2017/2018, a bunch of the data scientists working in this space, we got together and started thinking about risk a lot.

 2:54
 At the time, there weren't a lot of guardrails for AI and there wasn't a lot of oversight.

 3:00
 And so anyone who's worked as a data scientist at the time, you know, from 2015, 2010 to 2020, you could do kind of whatever you wanted.

 3:09
 And when you're working in a high-risk environment, when if you make a mistake in your prediction, if there's a privacy leak, if there's a bias, it could have a really big impact on people's lives, right?

 3:22
 It starts to bring to light the importance of policy and guidelines.

 3:26
 And so I started working on some internal guidances at the UN for data scientists to use to be able to communicate the risks of the tools that they're developing, uncertainty related to those tools, and then how we might be able to mitigate them.

 3:43
 And that led to my work in AI governance.

 3:46
 And so over the last four years, I led a research team that helped UN member states figure out how they wanted to collaborate together on risks and opportunities of AI.

 3:57
 So governance at the global level is a little bit different from governance at the state level.

 4:03
 Sure.

 4:04
 And the member states of the UN decide what they want to do.

 4:07
 Do they want to have a kind of agreement, a convention?

 4:11
 Do they want to have dialogues and meetings about it?

 4:15
 All kinds of...do they want to have a fund for capacity building on AI?

 4:19
 So, all kinds of different mechanisms.

 4:20
 And so our team for four years has been advising and doing research and trying to help the UN figure out how to best address that.

 4:29
 And then I came here to to the state.

 4:32
 Wow, that's incredible, incredible journey.

 4:34
 So you you're now here with New York State.

 4:36
 What were your first first impressions of New York State government and ITS specifically?

 4:41
 Well, I think I hadn't quite realized how big it was.

 4:45
 So that was really impressive.

 4:47
 I mean, it's a huge machine.

 4:49
 There's so many people that work there, there's so many agencies and I felt like the sense of responsibility towards New York State residents is very strong.

 4:58
 Yes.

 4:59
 And that's something that really struck me is that we're, we have, you know, taxpayer dollars.

 5:04
 We want to make sure that we're delivering the best possible services.

 5:09
 And how do we do that?

 5:10
 And in, in my case is how do we utilize AI to do that and make sure that AI deployments are helping people rather than the other way around.

 5:18
 So I also realized that there's a lot of really competent people in ITS, the team already coming in with the Chief AI office, we're about 15.

 5:30
 Most people have, you know, 15, 20 years of experience in this state.

 5:34
 And so they come with this amazing institutional history and knowledge that I really rely on.

 5:43
 And I feel that the expertise is not just technical, but also the fact that we're operating in a complex environment.

 5:49
 So it's a very complicated bureaucracy, right?

 5:53
 And so people understand processes and they try to figure out how to get things done and the different layers of stakeholders that we have in state.

 6:02
 And so that's been my impression.

 6:04
 Generally it's, it's a complex environment, very, very interesting to work in.

 6:08
 And some great colleagues.

 6:10
 Absolutely.

 6:11
 So we're talking about, you know, talented AI, chief AI office and all the amazing professionals that are working on this on this technology.

 6:21
 AI has been dominating the headlines for past few years.

 6:25
 We do touch on the subject with a number of our guests who are, you know, talented professionals here at ITS.

 6:31
 Can you give our listeners like a high level overview of just what technology we're talking about when we say AI?

 6:39
 Sure.

 6:39
 That's a really good question.

 6:40
 I think over the last few years, since 2023, so not even that long, we've talked almost only about AI all the time, every day, and that's mostly been generative AI.

 6:54
 So generative AI is a kind of a class of AI, like a subfield of AI that is focused on creating content.

 7:04
 And basically it stems from a field of AI called that used to be called natural language processing.

 7:11
 So it would be analyzing text and and speech.

 7:15
 And then in the generation element, it's generating responses.

 7:21
 And that's a part of AI.

 7:23
 And now the capacity that we have to, to generate various kinds of content is really strong.

 7:28
 But there's other kinds of AI tools, machine learning, which is what I tended to work on in my predictive work, which is to make, build different models to make quantitative predictions.

 7:41
 There's computer vision, there's robotics.

 7:44
 So there's there's a lot of different kinds of AI.

 7:46
 And what they have in common is that AI more generally uses data, creates models on top of that data to be able to predict future items, future elements.

 7:59
 So in the case of generative AI, it's basically predicting what would be the best response to your prompt.

 8:05
 So it's a predictive exercise.

 8:08
 It's just that it's presented in a different way than when I was running a model, which would be, here's some data, I want to know what this is going to look like 6 months down the line.

 8:18
 And then I would get a prediction.

 8:20
 The form of the prediction is different, but ultimately it's basically creating models, just like in statistics, you would do a regression analysis or something like that, but kind of at a larger scale capacity to analyze much more data, do much more computational analysis.

 8:38
 And so that's what AI is generally.

 8:40
 And so, because over the last few years, we've...it's become a tool that is in the hands of everyone, the importance of the, the technology has really become front and center in people's minds.

 8:53
 And that's when instead of having practitioners develop their own guidances, it became important for policy makers to be involved and say, how can we guide the development of AI in a place that is good for people?

 9:07
 Exactly.

 9:09
 So let's drill down a little bit more.

 9:11
 Let's talk about, you know, the good, bad, the ugly of this technology, the risks associated with AI.

 9:18
 I'm sure that's top of mind for a lot of our listeners.

 9:21
 It is.

 9:22
 It definitely is.

 9:24
 And there's a lot of hype, I think in kind of both ways.

 9:28
 I would say that the in like a lot of things today, there's polarization in the conversation in AI.

 9:35
 There's one side which is talking about, you know, existential risk to humanity and that it's really going to harm us in a in a massive way.

 9:45
 And there's the other side that says that AI is going to revolutionize society for the better.

 9:50
 You know, everything's going to be smoother, everything's going to be better, and we won't have to work anymore.

 9:56
 You know, so there's kind of two very strong sides and they're often together.

 10:01
 And so people keep being confused because they're kind of bombarded by this information all the time.

 10:06
 But I think the role that we have in state is to have a bit of a kind of middle ground and to say it's a powerful technology.

 10:14
 We have the capacity to utilize that technology for specific things to help, you know, solutions that we want, right?

 10:23
 We can guide the development of the technology where it would have the maximum positive impact and that's in the, you know, improvement of service delivery, reduction of administrative burdens and so on.

 10:42
 So we can only do that if it's safe.

 10:45
 And so there needs to be a risk mitigation strategy.

 10:50
 And so at a national level, we have risk management framework, the NIST AI Risk Management Framework, which is a set of standards that provide a lot of guidance in AI development.

 11:03
 So from the NIST AI Risk Management Framework, the New York State has published the acceptable use of AI policy, which requires state agencies to have several risk mitigation strategies.

 11:17
 And so before any AI tool is deployed for public benefit and inside the state, there needs to be a risk assessment.

 11:26
 And this...the categories of strategies for risk mitigation in the acceptable use of AI policy include human oversight.

 11:36
 So making sure that there's always somebody, a human being monitoring the AI tool that is not just working kind of independently and and without oversight.

 11:46
 Human in the loop. Human in the loop. Human in the loop is really important. Transparency.

 11:51
 So there's a disclosure we publish every year now the AI inventory to share the uses of AI for public benefit in the state.

 12:02
 And there's a lot of communication around the AI tools internally and, and how they're being utilized and how they're being tested.

 12:10
 There's a question of equity.

 12:12
 So it's really important for there to be accessibility around those AI tools, but also to be for there to be bias mitigation.

 12:19
 So the tool should not be working better or having better results for one group of the population or one group of the state population than another group.

 12:28
 And the reason why we have bias in AI technologies is often for a relatively simple technical reason that AI tools can often be trained on data that is not representative of the population, right, towards which the the prediction or the tool is aimed.

 12:52
 And so when that happens, there's the error rates can be really higher for some parts of the population than for other.

 12:59
 And that is what one of the ways that we we see bias in AI tools.

 13:03
 And so that's one of the things that we have to be really careful when we deploy AI tools to make sure that the underlying data and the models and the way in which we monitor those tools is representative of the New York State population.

 13:17
 Absolutely.

 13:18
 And for those of our listeners that are interested, a copy of that policy is available on the ITS website.

 13:24
 So I think you've touched on this already, but let's go in a little bit further.

 13:29
 How do you see this technology benefiting everyday New Yorkers while mitigating risk?

 13:34
 What areas do you think will see the most impact?

 13:37
 I think where we need to focus on as a priority, and I think a lot of different agencies in New York State agree with this, is where there is real bureaucratic friction.

 13:48
 So anytime that you as or us as New Yorkers, we are trying to get, you know, a document done or we're trying to go on a platform and get a certain service and we face different barriers, right?

 14:02
 That's where we're hoping to bring AI to help smooth that, whether it's making sure that we have common processes, making sure that there might be different layers of conversation and communication.

 14:18
 So one area of AI that's has a lot of support is the idea of customer service, right?

 14:26
 Could there be an easier way to update data, to summarize documents, to provide information to New Yorkers?

 14:35
 Sometimes websites, New York State websites can be out of date, but could there be an AI tool that kind of goes back, checks updated policies?

 14:44
 Make sure is that the websites are always up to date, that the information provided is always good.

 14:49
 There's certain chat bots that have already been deployed to say, if you're looking for basic information, you don't have to wait on the phone for an hour to speak to an agent.

 14:58
 You can have, you know, a kind of quick chat to sort of do some triage and see what are your needs and, and where could we best orient you.

 15:07
 So these are the kinds of things that would be useful for New York State employees.

 15:13
 There's a lot of benefit in reducing the burden of administrative work.

 15:18
 So freeing people up in where they're doing kind of rote work or repetitive work, having that a little bit more streamlined so that they can focus on tasks that are more analytical, more impactful, potentially even more creative, right? So that they can use all their state experience to to improve processes themselves.

 15:39
 And so we've seen in our own deployments at ITS of generative AI systems, and I'll talk about that a little bit more.

 15:47
 We've seen a lot of people wanting to use AI to summarize documents, to produce briefs, to analyze state policies, because there's a huge amount of compliance work that's done at New York State.

 16:01
 And so all of that work kind of reduces the cognitive load of of text production and analysis and can be also give us the opportunity to use AI as a thought partner.

 16:14
 So, bounce ideas off. Brainstorm.

 16:16
 That's actually for me.

 16:18
 I actually love writing and that's not the writing part is not my primary use of AI, but the thought partnership is a very strong use of AI for me.

 16:29
 I love brainstorming and having an idea and having, you know, different bullets and different points back.

 16:36
 Ideas come back to me and kind of going back and forth to refine that idea.

 16:41
 I think every person has different ways of learning and being productive and so they might have different ways of using AI tools as well.

 16:51
 But I do think that as a tool, a kind of collaborative tool inside New York State, it can also benefit a lot of a lot of State workers, right?

 17:01
 A tool in the toolbox.

 17:02
 Exactly.

 17:04
 So in, in terms of this implementation and at the governor's direction, an AI education training pilot program involving about 1,000 employees across New York State agencies was recently completed.

 17:18
 Can you talk a little bit more about the results and what do they indicate about the future of AI within the New York State employee workforce?

 17:27
 Sure.

 17:27
 So the AI Pro was deployed.

 17:31
 It was, it's a generative AI tool.

 17:33
 It's been developed inside New York State by ITS. It uses a large language model, which is Gemini, which is provided through Google.

 17:42
 So it's Google's large language model, but other than that it is really kind of customized and built within ITS for all New York State agencies.

 17:55
 And you know, we even have the capacity to to change the large language model if we want.

 18:00
 So it really the level of customization that we have, I think for me is has been really impressive.

 18:07
 In the pilot,

 18:08
 there's over 1,500 people that used it across 8 different agencies.

 18:13
 And there were a lot of really interesting results from the survey.

 18:17
 One of them is that 1/3 of the users had actually never used a generative AI tool before.

 18:22
 So they hadn't used ChatGPT or Claude or CoPilot and this was their first experience.

 18:30
 We saw a couple of things.

 18:32
 One of them is that those that completed the training had a better experience.

 18:36
 So through our partner, our training partner InnovateUS, we provided a training program that touched on AI literacy, prompt engineering and ethics.

 18:49
 And we asked everyone to complete the training because we felt that it's a really good opportunity to make sure that those that are using the tools have the different literacies and skills and also understand the ethics and the risk.

 19:07
 And I continue to think that doing this kind of training when we're deploying AI across the state is, is really important and critical.

 19:15
 So those that completed the training had a better experience.

 19:19
 We also saw that confidence in AI increased during the pilot.

 19:23
 So as people became more comfortable with the AI Pro tool, their perspective of AI increased.

 19:32
 So the fears that people might have had around AI tools were higher when they weren't using the tool.

 19:39
 As they gain more experience, the fears came down.

 19:42
 And so that's a really important point for me to say we need to have these tools in our hands so that we can demystify them.

 19:49
 And that's something that I've, I've actually always felt even in other kinds of AI, data science and predictive analytics, there's always been this kind of idea that AI is a black box.

 20:02
 It's kind of magical.

 20:03
 It's only for certain people.

 20:05
 I know we have a lot of highly technical people in ITS, but that's not necessarily the case across all of New York State.

 20:12
 And I don't think that you need a computer science degree to be able to familiarize yourself with AI and especially that it's such an important tool and kind of

 20:23
 innovation in our society that's having such broad ranging effects. If you don't, if you're not able to master that tool and if you're not able to use it, then you're kind of left with just what you read about it and what other people say about it.

 20:40
 And I think it's a really great opportunity with these deployments of AI that are actually quite accessible for anyone to use to make up your own mind around what the tool should do and and how it should be developed.

 20:53
 Sure.

 20:54
 And also be part of the conversation.

 20:56
 I think everyone in New York State should be part of that AI conversation as well.

 21:01
 That's fair.

 21:03
 So we talked about a little bit already, but the first ever New York State AI policy was published in 2024.

 21:10
 It provided general guidelines for the use of AI within state government.

 21:14
 As you mentioned, one of your stated goals in this new role was to refine and improve this policy as this technology evolves.

 21:21
 Can you speak to that a bit more?

 21:23
 Yeah.

 21:23
 So AI as a technology is going to continue to evolve.

 21:28
 As we know there's a huge amount of investment that's being put into the development of these technologies.

 21:33
 And so the kind of big AI companies are developing at a very rapid rate and also smaller companies are, are kind of part of the mix and, and pushing AI into new directions.

 21:46
 So that's one part of the landscape.

 21:48
 The other part is that as these technologies evolve, our uses of AI also can kind of change and evolve, and our capacity to integrate AI into our workflows will increase because AI is not necessarily a magic bullet.

 22:03
 So actually using the tools and bringing it into the processes in a way that makes sense and actually has positive results actually takes a lot of effort.

 22:13
 It's not going to be easy for every office to do.

 22:16
 It's going to take some thought.

 22:17
 And so over time, we're going to see an increase in adoption and an increase in useful use cases so that it's not just tools that, oh, do cool things, but tools that actually make a really big difference in New York State.

 22:34
 As that happens, our policy is going to have to adjust, right?

 22:37
 One of the first focuses that we have in our team is to make sure that there is clear implementation guidelines for the policy.

 22:45
 So a policy is always kind of high level and it puts together some principles and some guidances in terms of making sure that we check for bias and so on.

 22:56
 But there needs to be more technical detail to really inform the work of the agencies.

 23:02
 And so in our team, we started a DCT tour.

 23:06
 So we're meeting which with each DCT team to understand their deployments of AI, if they have some, with the agencies in which with which they work, what some of their pain points might be, what some of their interesting use cases might be.

 23:21
 And we're going to integrate that also into future implementation guidelines and other elements of our team's work plan to make sure that we're supporting their AI deployments. Right. Fostering collaboration.

 23:34
 So we have data-based policy in place for for this technology.

 23:39
 Yeah, definitely.

 23:41
 So we've talked about this also a little bit.

 23:44
 Accessibility, customer service, human-centered design.

 23:48
 These are high priority topics within ITS right now.

 23:52
 How do you see AI specifically affecting these areas?

 23:56
 Well, I think the accessibility element is it's making sure that when we're deploying AI tools, they are accessible.

 24:05
 So I think that's the first element.

 24:07
 It's kind of a core principle of the AI tools that we're deploying is making sure that people can use them, and that's why also having them within New York State and having them paired with training is so important.

 24:24
 The dimension of customer service, I think there's a strong use case for that, increasing the and improving the quality of information that's disseminated, increasing the speed at which people are able to receive services and obtain the information that they need.

 24:42
 So all of those kinds of efficiency gains, I think from a customer service perspective are really relevant and useful.

 24:50
 And I've lived here in New York State, you know, there's definitely processes that are challenging that take a lot of time.

 24:56
 You need to take a day off work to get some card or some form.

 25:00
 I'm not going to name names, but but it takes a while.

 25:04
 And so for me, those are the primary use cases where we have the pain points is where we should be trying to deploy this technology.

 25:11
 That is fair, that is fair.

 25:14
 Eleonore, we're coming to the end of our episode, sadly.

 25:16
 Before we let you go, is there one, There is one final question we ask all our guests.

 25:21
 What are you looking forward to the most this year?

 25:24
 And it absolutely does not have to be work related.

 25:27
 So in my free time, which I haven't had so much of in the last six weeks, but I'm, I'm definitely planning to is I play cello.

 25:35
 Oh my goodness.

 25:37
 And I love music.

 25:38
 I love classical music.

 25:40
 If, if there's an orchestra in New York State, I will join it.

 25:42
 I don't know if it exists.

 25:44
 Oh, absolutely, absolutely.

 25:45
 There there is.

 25:47
 Call me up.

 25:49
 I'm currently still part of the UN Symphony Orchestra.

 25:52
 The the staff have have a Symphony Orchestra.

 25:55
 And this year we're playing a really cool Symphony in June called the New World Symphony by Dvorak.

 26:03
 And he was a Czech musician and he composed it.

 26:06
 He did a one year fellowship in the United States and he composed this amazing piece of music and it's really beautiful.

 26:14
 The 4th movement actually inspired John Williams when he composed the music for Jaws.

 26:19
 So it's very iconic.

 26:21
 And so I've been practicing it and I'm really looking forward to performing this in June.

 26:25
 Oh my goodness, I'm sorry.

 26:26
 Other guests, that might have been the best answer we've ever gotten to that question.

 26:31
 Eleanor, thank you so much again for talking to us today and sharing your vision for AI implementation in New York State.

 26:38
 Thank you.

 26:38
 It's been a pleasure.

 26:39
 Thanks for having me.

 26:40
 Thank you for listening to It's a Tech Podcast.

 26:43
 For more information about ITS, visit ourwebsite@its.ny.gov.