Catch Investigator Grant Discrepancies, Budget Drift, and Accrual Risk with AI

Welcome everyone to our second webinar in the Practical Applications of AI series that we're running. Today we are gonna be speaking a little bit about reimagining investigator grants with AI.

‍

I'm Jeff Yahnke. I'm the director of customer operations here at Condor, and I'm joined today by our VP of product, Nim Fox. We're really excited to talk to you all, both about some, challenges and hear from you all about some of the challenges of managing investigator grants payments given what a large component of your clinical trial spend those particular items are.

‍

Teach you a little bit about AI in terms of what you can do today with tools like Copilot to begin the analysis, and then also show you a little bit about, what we've been building here at Condor that is purpose-built and, set to change the game when it comes to investigator grants analysis and reconciliation.

‍

So I always bring it back to this slide when I lead these because we had a really great panel at the twenty twenty-six ADFO National Conference, and the big takeaway, and I'm sure a lot of you feel the same way, is, you know you need to be doing something with AI. You know that it's powerful, but you just don't know where to start.

‍

And the question that we got that kind of prompted this webinar series is looking to us to share practical hands-on examples of how to use AI in clinical finance. And so that's the whole purpose of this. If you've got questions throughout the presentation, please feel free to drop them into the chat.

‍

We'll try to answer them live, but if we for some reason don't get to your question live, there's gonna be plenty of time at the end to talk through those. But I'd also like to hear in the chat if anybody's got any other ideas for topics that they'd wanna hear about. We covered CRO change orders in the first one.

‍

We're doing investigator grants today, but would also love to hear from you all what topics would be most impactful to you today what we're gonna talk about, as I said, is investigator grants. So we're gonna review an investigator grant payment report end to end using tools that you have available to you today.

‍

We're also going to surface overpayments, off-contract rates, and accrual gaps before they become true ups. We're gonna validate those findings with EDC and with your contracted rates, and then draft follow-on language back to the CRO, requesting support for unmatched payments. So packed schedule. Feel free to follow along.

‍

I think for the, the practical work that I'm doing in Microsoft Copilot, the prompts that I'll be using will be included as part of a blog post that will be posted later today. So don't feel like you gotta have your, Microsoft screen capture tool ready to quickly capture anything that I'm doing in Copilot.

‍

We'll share all of that to all of you following the webinar.

‍

And just to give you all a little background about Condor in case you're new to Condor as a company we are the AI platform for life sciences, R&D, finance, and operations. We partner with the big four accounting firms, and we are trusted by leading organizations in the life sciences space.

‍

Some of our stats is our FP&A tool can lead to ninety percent forecast accuracy on the accounting and accrual side, seventy percent faster month to close, and then up to thirty percent budget savings on vendor management and overspend catches. So let's talk a little bit before we dive in just to set the stage.

‍

Why AI? Why now? And revisit the framework. I covered this during the first webinar, but essentially, we think about AI in three different stages. The first stage is chat with a document. And as we've talked with folks throughout the industry, I would say the majority of people are in stage one.

‍

Basically, you've got your ChatGPT, your Copilot, your Claude. You might be uploading a document, but really you're using that window to just chat with a particular document and surface insights about that document, but not necessarily getting a lot of context outside. The benefits of this is that it's fast, right?

‍

And it's easily accessible. Anybody can use it. Stage two is workflow-embedded AI. It's AI inside the tools that you already use, so it's built on the data that you might already have. So for example if you're using like NetSuite as your ERP and they roll out an agent inside of NetSuite that allows you to chat with the data that is in your ERP system, for example.

‍

There's a benefit to that in that it's built into the tool, so it obviously can read all of the data within the tool. The third stage is purpose-built AI. So domain models that know everything about your clinical trial and know everything across all of your systems, so that way you can really leverage AI to get deep insights into your own data.

‍

So this might be a system that connects AI that, that potentially connects your ERP, your EDC, your CTMS all of the various systems, and is built on top of what we call a knowledge graph or a deep understanding of this-- the domain, the, the industry, so that it understands all of those data points and how they all tie together and relate to one another.

‍

And the beautiful part of this is that- AIs with this type of technology are evidence-backed, repeatable, and audit-ready because they tie to all of your systems, they have the knowledge, the understanding, the context needed to make repeatable and repeatable and very intelligent conclusions about your data, your trial, where it lives, how it works.

‍

So yeah, as I mentioned before, last session, we tested this kind of three-stage approach on CRO change orders, and today we're gonna test it on what is likely the largest individual line item in your clinical budgets, which is investigator grants. So first, let's talk about the investigator gr- grant problem.

‍

And before we dive in, we've got a quick poll. We're curious about how all of you manage investigator grants today. CRO or site payment vendor reporting, do you rely on manual spreadsheets and trackers to try to recalculate? Are you using clinical activity data such as EDC or a combination of the above?

‍

And there's a poll that should have just popped up on your screens. Feel free to make a selection there. That'll help inform how we drive this conversation. We'll give everybody a, a couple seconds here.

All right. Sounds like a combination of the above. Glad to hear that.

‍

Let's let's move on to the next slide here and talk about investigator grants and why they're so important. So first of all, nearly half the budget flows through investigator grants. So on average site payments related to either administrative fees or patient visits account for roughly forty-eight percent of total per trial cost.

‍

The average investigator grant cost per patient across all therapeutic areas is about seven thousand dollars. And on average oncology runs closer to eighteen point-- eighteen thousand seven hundred dollars per completed patient. But in, in reality, depending on the phase, depending on the specific indication, depending on the protocol, we all know that those costs can oftentimes be astronomically larger.

‍

You're approaching six figures per patient on some very in-depth rare disease protocols and oncology protocols. And just the volume of data. Because everything is being tracked at a transaction level, those patients can generate tens of thousands of line items of individual transactions that all have to be verified for accuracy, both against whether or not that activity actually occurred, whether or not the site earned that money, through like screen fail caps and ratios, things like that.

‍

And then also validating that the cost being invoiced by the site is correct. Spreadsheets just can't scale with that volume of data in a way that more modern systems can.

‍

Another issue is with regards to payment terms to sites, right? How long a research site waits to be paid for work already performed. So monthly payment terms, on average, a site's getting paid about 45 days after an individual visit is processed. On quarterly payment terms, it can be three times the wait for the same work, or on average, about 137 days.

‍

But the problem is that these sites oftentimes are small businesses in and of themselves, and they just don't have the cash runway to keep operating while they're waiting to be paid for work to be done. 43% of sites report three months or less of operating capital in the bank. So quarterly payment terms, depending on the volume of patients they're treating, could put the site under.

‍

40% of sites that drop out of studies, the, the reason is overpayment delays. Basically, they're not getting paid in a timely fashion or per the terms of their contracts. And then also 23% of sites report payment delays of 90 days or longer. It's not just the payment, it's the accuracy of the payment as well.

‍

And we all know that sites who don't get paid aren't happy, and sites that aren't happy aren't gonna participate in your next clinical trial.

‍

It's site relationship and enrollment risk, and another reason why grants are just hard to get right. First, the activity itself and the money live in different systems.

‍

Sites will negotiate a CTA or clinical trial agreement that contains a budget for how much they're getting paid per visit, and that gets tracked via like a contract management system. On the other side, the actual patient activity that's taking place is logged in an EDC system, and very rarely are folks who are managing a clinical trial manually reconciling those two against each other.

‍

The EDC knows when visits happened, finance only sees the invoice and the payment files. The second challenging thing is that every site contract is a bit of a snowflake. The, the rates themselves, the invoiceable procedures, what they're charged for on scheduled visits, holdbacks, screen fails, caps and ratios, currencies, overhead percentages, all of these items are buried in PDFs.

‍

And frankly for those of you who've seen some, some of these PDFs are like photocopies of photocopies, so the version that you get is practically illegible. And the format of them can vary widely as well. So it's just a burden, an administrative burden to manage and tease out what the right terms even are if you have all the contracts.

‍

The third thing that's difficult has to do with protocol amendments. We all know that protocol amendments especially major ones, can change obviously the protocol, but also the underlying cost per visit, the administrative fees the payment logic. And understanding when those new contract amendments are effective for a site impacts the accuracy.

‍

Because you might be paying a site correctly for a visit that-- or a visit priced on a previous amendment, but the new amendment is updated, and then all of a sudden that site might be feeling short-changed. And then the last piece, which is a massive issue, is that invoiceables accrue in the dark.

‍

And what I mean by that is a lot of times procedures, screen fails, unscheduled visits, they're billed when they're incurred. And if nobody's actually tracking the volume of activity that's taking place, they may surface all at once as a true-up at close out. And remember that, because I have a story to tell.

‍

So how do sponsors manage these grants?

‍

They're so complex. The answer is oftentimes they outsource to a CRO or a payments vendor. And that's often the right call. Administratively, they're armed with the workforce to manage these site payments, to manage these costs. And they're doing it for dozens or hundreds of trials as an organization.

‍

And it's often the right call, but it's not a complete one, and here's why. First, outsourcing has its benefits. As I said, the, the relationships and the budgets are negotiated by people who that's their job, their full-time job is to do that, and they're pretty good at it. They also manage sites internationally, so you don't have to worry about currency conversion, VAT, managing the tax forms, managing the FX gain loss, Sunshine Act reporting.

‍

And it's also great because, it's a sig- substantial headcount in oftentimes, and that's headcount that you don't have to hire for a function that you don't wanna build. You wanna focus on, the success of the trial, not necessarily on recreating a wheel on site payments. And the other nice thing is, of course, just the scalability.

‍

You're in charge of managing the site payment vendor, whether that's your CRO or a third-party vendor. You don't have to worry about managing dozens or even hundreds of site... individual site relationships as well. But the downsides of this are that you don't necessarily have line of sight into what's actually taking place.

‍

You're relying on the vendor to report to you what's been paid and trusting that's accurate. Payments having worked at a CRO myself, payments are often a low margin afterthought. They're a pass-through. So there's not as much care or detail being given to them as maybe some of the direct fees, invoicing or similar, 'cause that's where CROs and other vendors make their margin.

‍

Quality control. You're outsourcing not just the payments themselves, but the payment approval, the calculations, the reporting to a vendor that is incentivized to tell you that everything's operating smoothly. And at the end of the day, your auditors and board are asking you about the numbers. They're not asking your CRO, they're not asking your your payment vendor whether or not they're accurate.

‍

It's up to you to manage that relationship. So at the end of the day, you're able to outsource the work, but you can't afford to outsource the accountability.

‍

And here's a couple stories about when convenience wins, but, kinda turns into a pretty messy situation. The first one is the auto-pay trap. A top five global CRO a couple years ago actually replaced its entire investigator payments team with an automated payment platform. Essentially, if the site sent an invoice to the CRO auto-paid it.

‍

But what started happening was sites realized that they were getting paid for things that had not happened, that payments were triggering in the system before a patient had even enrolled, and were reaching out to the CRO saying, "What was this check for? We received a check from you, thank you, but we don't have any payments enrolled on this study. Why are we being paid for patient visits that haven't happened yet?"

‍

The confusion took months to unwind. Obviously, sponsors were caught off guard 'cause they had outsourced that accountability to the CRO, and the CRO let them down. Another one is the closeout surprise, and this kinda goes back a couple slides to what I talked about, invoiceables accruing in the dark.

‍

A midcap oncology sponsor closed out a trial, and at the end, the CRO sent them a bill for five million dollars that the CRO had not been tracking invoiceables, and a bunch of sites invoiced them for all of the invoiceables that had taken place over the course of the study, but waited until the end of the trial to do so.

‍

The CRO did not track or accrue this invoiceable activity, so it was a surprise to the CRO as well. The money had been spent for years and was just not being reported anywhere, and the sponsor was the last to know. And think about five million dollars is a lot of money. And that's money that, that was likely already forecasted to be invested in a subsequent trial or a new program or, GNA even, and that had to be reallocated towards a liability that they were completely unaware that they had.

‍

What I would say is that automation without reconciliation just makes the wrong payment happen faster.

‍

A question, another question that we have for all of you is, how confident are you that in your current process you can quickly identify over-billing, under-billing, duplicate invoices, or missing payments? And we've got a scale there. Very confident, somewhat confident, not very confident, or we don't even have consistent visibility.

‍

And on that last one I know that sometimes the number that you're receiving in your CRO reporting, for those of you that use CROs, could literally just be a flat number. We've incurred $2.5 million of grants, but no visibility into what patient visits make up that data, what site admin fees make up that data. So curious to hear for your thoughts.

‍

All right. Looks like most people are somewhat confident or not very confident. I'm glad to hear that... it seems like a couple folks don't have consistent visibility, but I'm happy to hear that's the minority of you, 'cause I know that this has been an ongoing problem for a lot of years.

‍

If anybody else has any stories about, you know, challenges with their payment vendors or CROs, feel free to drop those in the chat. We'd love to hear about them. But yeah. So let's get to the the fun part of the demonstration, the live demo with Microsoft Copilot. So I'm gonna go ahead and with Microsoft Copilot here, what I've done is I've dropped in a payment file.

‍

Is it gonna make me log in? Let me see if I can pull this up a different way. Standby.

‍

I've dropped in a payment file. What you can see here is a summary of the different payments that have been submitted to sites for various screenings, invoiceables, visit payments, the payment run, essentially what the invoice amount was to the sponsor. We've also got payment detail, which if you're able to get ahold of this, this is kind of the holy grail that unlocks the ability to do a more granular reconciliation.

‍

But essentially all of the CROs maintain something like this. They have to for their own reporting. It's just a matter of whether or not they're sending it to you. But essentially, something that tells you what sites are being paid and for what, and the dollar amounts that are being paid. And then as, on a separate tab, you've also got the invoiceables and startup fees as well, so all of the administrative fees.

‍

So if I go back into Copilot, I've loaded this file into Copilot, and I've dropped in a prompt here, and the prompt says, "I'm the head of clinical finance at a biotech company. Attached is the investigator grant payment report from our CRO for our phase three study covering the last two quarters.

‍

Please analyze it and give me total payments by site and month, highlighting the five highest paid sites, a breakdown of visit payments versus procedures versus invoice of all pass-through line items, any payments that look inconsistent with each site's contract budget or with the visits, or with the visits patients actually completed, what we should accrue for work sites have performed but not yet been paid for, and five questions I should ask the CRO before approving the next payment run. Format the output as a brief memo I can share with my CFO."

‍

So a couple things about the way this prompt is structured. One is, I'm being very specific about what I'm asking for. The reason for that is, these LLMs are really good at generating what I would call credible looking documentation.

‍

But at the end of the day, if you're not being specific about what you're looking for, it's going to make something up, and this is what kind of gives you the ability to be somewhat repeatable with this, is having a prompt that is very specific. The other thing too is I'm giving it context on how I want it to think.

‍

So I'm telling it who I am. I'm the head of clinical finance at a biotech company, and I wanna deliver this to my CFO. That context about the audience, what frame of mind that the model should be taking, and also what who the intended audience of the output is gonna give it more context on how to format it.

‍

So I need to think about this as if I'm the head of clinical finance, and I'm gonna be sharing this output with the CFO, so that way it knows what level to keep the data at and also, how to think about the problem. That's where people oftentimes get less than ideal results, is when they don't give the model that type of context of what persona the model should be embodying, and then it approaches it from a place that it thinks is appropriate but might not be.

‍

So I'm gonna go ahead and click send.

‍

And it's thinking about it. Reading the Excel file.

‍

And now it starts to generate it. It's putting out a really nice executive summary, basically saying that the CRO sent two hundred and seventy thousand dollars. Payments are mostly weighted towards startup activity and pass-through costs, reflecting a study still in early enrollment. No obvious duplicate payments were identified, but several reflect CRO follow-up.

‍

So then you get, your top five sites by total payments, and it's outlining all of the different sites that it's paid. It's telling you the payments by month. The payment profile is uneven, with large spikes in December and May, driven primarily by startup fees, IRB pharmacy costs, and batches of delayed procedure payments.

‍

Here's the mix of the payments. You get to see what your operational activity is, so mostly a lot of visits and screen failures, but also a lot of invoice-able pass-through line items. A little bit of procedures. Here are the payment inconsistencies that it's flagging, basically saying that there is a lag from four to five months for these sites between the date that they actually performed the visit and when they were reimbursed.

‍

There it caught a couple things about where it thinks that payments were maybe incorrect, but it's basing that all on just what it's inferring from the files instead of actually understanding the context. So in this case, it thinks that this biopsy and imaging reimbursement is incorrect because it was paid before the screening visit itself was reimbursed.

‍

Multiple ECG charges appear separately months after associated visits. What you're getting here is a really solid summary of what that file tells you, which to be fair, if you were looking at that file on its own, you likely wouldn't be able to glean these types of insights as quickly as this was able to be generated.

‍

The other thing, though, is if you were to run this again, like you'd likely get the same numbers, but the, the type of analysis might vary a little bit. That's also something to be cautious of. And some of these conclusions that it's drawing that it seems pretty definitive about may or may not actually be issues because it doesn't have the full picture.

‍

So if I jump back into the slideshow here it's... It explains what I what I just said, is that I think general purpose AI can be general- genuinely useful. That's something that you can do today. If you're getting a payments file from your CRO, you can go into Copilot and generate, a pretty clean memo that summarizes at a very high level what the status of your payments are and allows you to flag certain possibility or certain possible issues with your payments file.

‍

You get that fast, readable summary, clean per site totals and top spender rankings. It's catching the obvious anything that's obviously duplicated or outliers. You get some good starter questions, and again, in just a, a few seconds, you're getting that CFO-ready memo with no procurement needed. It's likely something that you've already got in your toolbox, whether, again, it's Copilot ChatGPT, Claude, whatever you might be using.

‍

But here's where it stops. It can't see the EDC, so it can't tie a payment to a visit that actually happened. It doesn't know your contracted rates, so there's no way for it to validate that, a site might be billing you for a screening visit that has gone-- or a screen fail visit when it's already gone over its cap or its ratio.

‍

It doesn't know holdbacks. It doesn't know the overhead. The accrual estimate that it generated is just an extrapolation of past payments. It's not measuring against actual activity. It can't tell an early payment from an unearned one, so it's flagging those biopsies as saying you might have paid this a little earlier than you needed to, but it has no idea whether or not that biopsy was actually performed.

‍

And again, there's always the risk of a different answer every run, and there's no evidence or audit trail behind any of them. These, these conversations that you're having rely on the context that you're giving the model in that particular window. There's not a lot of shared context. So essentially, it reads the payment file, but it cannot read the trial.