When Excel Breaks: Choosing Clinical Finance Software with PwC

Bethany Lavery (PwC): Good morning or afternoon, everyone, and thank you for being here. Before we get started, some admin. As you can see on the screen, in order to get CPE you'll need to be on your laptop and able to answer the polling questions. You'll see a few questions pop up to start, and more later in the session.

With that out of the way, we're excited to talk about how clinical stage biotech and pharma companies can transform finance operations using clinical finance software. I'm Bethany Lavery, a director in our health industries assurance practice at PwC, and I'll be your moderator. I'm here today with Jennifer Kyle, CEO and founder of Condor Software, and Holly Reeves, Partner and Health Industries Emerging Company Solutions Market Leader.

Holly, why don't we dive in? Do you want to tell us more about some recent observations on clinical finance?

Holly Reeves (PwC): Thanks, Bethany. I'm excited to be here with everybody today. It's an important topic for most pharma and life sciences organizations. The world at this point is moving toward how you better streamline and systematize all of your operations, and there's no better time for a system to be put in place to track your third-party research and development expenses, your budgets, your forecasts, and your budget to actuals.

Most companies I've seen in my career track this in Excel files, going back to contracts and SOWs. So I'm excited to sit with Jen and talk through how you think about implementing a system: what questions to ask, what to look out for, and what the benefits are. Sitting as an accountant, connected to both the finance world and to all the key stakeholders in an organization, finance plays a critical role in this whole clinical finance transformation.

Bethany Lavery: Thanks, Holly. Jen, I understand there are a number of macroeconomic trends impacting clinical finance right now. Could you expand on those factors and how they're causing the role of finance to evolve?

Jennifer Kyle (Condor): Sure, and thanks for having me, Bethany and Holly.

First, this is a really exciting time for finance and accounting people, because inherently we want to add value to our organizations. We want to be a business partner. In the challenging market we're in today, this is an opportunity to step up and be exactly that. That's what we're seeing from the companies we speak to and from former colleagues of mine. For those who don't know, I used to be an auditor and worked in biopharma for many years doing this on spreadsheets, so I get it.

What we're seeing is a lot of R&D financial transformation, because it's no secret that the capital environment is tough right now. If you're a small company trying to go public, it's hard. If you're already publicly traded and flush with cash, you're still thinking hard about your capital allocation strategy. With a lot of new technologies coming out at the same time as FDA uncertainty, we're trying to figure out how to better partner with our R&D function, and how to help the board and the CFO make better decisions about which programs get capital.

If you're a small company, it's about extending runway, helping your clin ops team negotiate better budgets, and getting to better financial reporting and clinical accruals. With everything tight in the capital markets and organizations trying to do so much on a shoestring, it's time to get creative. That's when you start asking questions outside the normal box: how can I bring something to my organization that saves us money, gives us financial visibility, and creates a win with the CFO, the CAO, and the board around SOX compliance and taking our close from 15 days down to one? It's an exciting time for anybody who wants to do more with less.

Bethany Lavery: Thanks, Jen. Holly, how are you seeing finance being impacted by these trends?

Holly Reeves: Jen mentioned she used to be an auditor - I'm still an auditor at the core, and that's my day job. I get out there and meet with others, so I'm seeing this from the inside as well as through a lot of conversations. At its core, Jen hit it right on the head: how do you get out of manual Excel files and into a system so you have real-time reporting that adds value to the organization?

There are a number of things accounting teams can be doing. How do you get to real-time reporting on budget versus actual, and on the forecast? If you're a clinical person, you need to make sure your vendor is getting paid on time. But money is tight these days, so you also need to make sure you're paying for the right value and that the organization got what it thought it was getting before some sort of progress billing arrives. And how are payments getting approved? There's a huge link these days between payments, reporting, and actual clinical progress, and finance is the function connecting those dots.

At the top, as Jen said, CAOs, CFOs, and decision-makers need this financial information constantly. It's a stream of thought all the time. The best way to push a button and have all the information at your fingertips is to be out of Excel files and into something that lets you put the right policies, processes, procedures, and rigor around the reporting the organization needs.

The focus differs by company. If you're more of an R&D shop or an emerging growth company churning through R&D, money is tight and financing is hard, so how do you have the right information available for leaders to prioritize? For our big pharmaceutical clients there's a lot of pressure on ROI: how are you spending money on research and development to create shareholder value? There it's critically important to have the right information at your fingertips to support decision-makers.

Outside the pure finance function, and a little further from my wheelhouse, there are all sorts of regulatory reporting metrics these days, and finance folks are often responsible for herding the cats and dogs and getting reporting from the CRO or the CMO. If you can put all of that into a system and push a button, your processes become a lot more efficient and effective.

Overall, the finance team is the glue between a number of stakeholders and carries the burden of maintaining information, financial and otherwise. As companies evolve, it's worth thinking through when the right time is to move off Excel into a real system with the horsepower to do this reporting in a more streamlined way. I'm looking forward to talking through the questions to ask, the things to think about, and the regulatory environment - and of course I'll give my auditor lens on how auditors get comfortable with the move from Excel into a system.

Bethany Lavery: Jen, anything else you're seeing in your role?

Jennifer Kyle: I echo a lot of what Holly said. Holly, remember the days when all you had was whatever the CRO reported? You're about to file your 10-Q, it already took you 30 days to come up with your clinical accrual, and then right before the filing you get a confirmation from the CRO with a completely different number than the one they gave you a couple of weeks earlier. That was what we were used to, and there was no good way of doing it, so we were all being creative about how to get comfortable - across different CROs, different therapeutic areas, and different phases, all of which bring their own complexity. It's fascinating that this is no longer the norm. It doesn't have to be the case anymore.

Holly Reeves: I remember those days. You got draft reporting for the entire quarter on day five, and that was only draft. The final reporting came in on day 30, three days before you wanted to file, with totally different estimates.

To your point, I do think CROs and CMOs have much better systems and processes in place now and are tracking more in real time. The question is how that leads into companies having similar systems, so they can take that information, put the right rigor and controls around it, and keep up with the reporting. The finance transformation in this area has certainly evolved over my career.

Jennifer Kyle: I'm going to poke a little fun at you here, Holly. There are three areas where finance can be a more strategic partner in these organizations. One is audit. Two is reporting and analysis. Three is financial decision-making, and how you bridge your R&D function into your finance function.

On audit, we see a lot of organizations, especially emerging biotechs, that have a critical audit matter around their clinical accruals. We all know auditors ask a lot of questions and do an amount of testing that seems never to end, plus the documentation around that testing. When you partner with your auditors and have the conversation - how do we improve our financial estimates so you don't have to do all this testing and don't bill us so much for it, how do we all get more efficient - that can be a good thing. Work with your internal audit team and partner with your external auditors. At the end of the day you want to be valuable to your organizations. You don't want to be the pain-in-the-neck auditor nitpicking everything because the PCAOB says you have to.

Holly Reeves: Fair. I'm going to poke fun back at you, Jen. I always say this isn't an auditor thing: management is responsible for financial reporting, management is responsible for their internal control structure, and management is responsible for their numbers.

With that said, it's critically important that whether you're in Excel files or in a system, you've aligned on the key risks in that area and on what management's controls are to make sure there are no errors in the financial statements. Then, whether you're testing controls or getting comfortable another way, auditors need to get comfortable with all of that. Alignment across the board is critical. But I always flip it around: management is responsible for the financial statements. Sitting in the finance organization, you're responsible for the completeness and accuracy of that information and for reporting it accurately.

Jennifer Kyle: Agreed. Fair enough.

That bridges into reporting, financial analysis, and financial decision-making. With the technologies available today, it's much easier to bridge clinical operational data into the contracts and the finance side, and to bridge the understanding between the two. Our clin ops colleagues don't understand finance - that's not their job. And it's very hard for those of us in finance and accounting to really understand what's happening operationally in a trial, why it's happening, and what the financial impact is.

Whether or not you use technology, the best practice I see across the board is finding ways to collect clinical data, because you can vouch for it. You can vouch how many patients are enrolled. You can vouch how many sites are active. If you draw in those data points and bridge them into the financial contracts - the big budgets with your CROs and labs - you have everything you need to produce an independent analysis: here's what I'm seeing, here are our estimates, and here's what we project them to be. That exercise, whether in technology or in Excel, is what we're seeing more and more. When I was doing this in Excel five, six, seven years ago, it was all about getting comfortable with what the CRO told me and asking clin ops whether it looked right. Now we have a bigger seat at the table by asking for those data sources.

Bethany Lavery: That's really helpful, and I'm an auditor too, but I'll remain the neutral party. This is a good segue into what companies should be thinking about in terms of the right time to move off a manual process. What are the key triggers to look for when considering a software solution?

Jennifer Kyle: I'll break the triggers into two buckets: an operational bucket and a red flag bucket.

Operationally, the first trigger is when your spreadsheets break. You see it because there are so many tabs that updating them all becomes hard, and you forget that one cell links to another cell in a tab over here, and nobody catches it in review even though you had trends and analytics set up. The first time that happens, it's a flag. The second time, you have to do something about it.

The second operational trigger is scaling. This may be less prevalent today than three or four years ago, though it's true for some indications. You go from a couple of trials, you get positive top-line results, and now you're expanding. You ask whether the process that was good enough can scale across the whole portfolio, and how many FTEs you'd need to hire to make that work.

Then the red flags, which is when something bad happens. One: your auditors tell you there's a significant deficiency in your control environment, or a material weakness, or that you're in a critical audit matter position without good processes, so you're high risk. That's a compliance trigger, because nobody wants to go in front of the audit committee and defend a material weakness.

Two: you get overbilled, or you get a big unexpected bill from a vendor. You weren't anticipating a $5 million change order true-up, and you say, this can't happen again - especially in today's capital environment when we need to extend runway.

Three: reporting gaps, which ties back to compliance. Your CRO gives you something, you're comfortable with that estimate, and then it's very different right before you file, and you can't figure out where you went wrong or get comfortable with the number. That's when it's time to think about a better process, and maybe about bringing in other tools.

Holly Reeves: Let me add to that. You hit a lot of good points, and I'll add one.

On scaling: when you go from a Phase 1 trial with 10 sites into Phase 2 and Phase 3 where you could have hundreds of sites, it's a question of tracking. Do you want to hire many more people to do it, or put it in a system where you lower your risk? As you mentioned, links can break, and you're far less susceptible to manual error and linking error in a system.

From the marketplace, I hear a lot of companies asking, before I get into that huge trial managing all these sites and a different contract at every site, how do I get something in place to track it - not just for reporting and paying vendors, but for budget to actual and forecasting? Where are we today? How many patients have been enrolled? How costly is it? These trials go way over budget, and being able to tell people in a timely way whether it's over budget, by how much, and what SOW captures that, matters. Finding the sweet spot - where the trial or the number of trials is costing you more in manual intervention than it's worth - is the critical moment to get onto a system.

The other important point is the capabilities of these systems, and I know they're being built out right now: how do the systems forecast in real time? Everyone is focused on cash and spend, and having real-time reporting at a CFO's fingertips is hugely beneficial.

Bethany Lavery: Thanks. If companies are sitting here today thinking these points are all relevant, what are the next steps for evaluating clinical finance software?

Jennifer Kyle: Number one, get very clear on where you are as an organization and what you need to improve. It goes back to the beginning of the conversation. Are you scaling? Do you have a compliance risk? Do you need better predictability on your financial runway? Do you want better management of vendors and change orders, which comes back to how much money you'll spend over the next quarter or the next four quarters? Usually all of those triggers are present to some degree, so identify the strongest one and get very clear on it.

If your goal is compliance because you're going IPO, or because you've transitioned from 404A to 404B, then you get clear on what to evaluate. Do they have a SOC 1? How do they help with the process? What is their internal process, so that I can trust the information coming out of the software? You'll want to drill into those areas.

If it's about specific reporting or forecasting requirements, you'll want to understand how the accounting side of the solution translates into the forecasting side. One of the nuances in biotech is that you have study budgets spanning three years alongside annual corporate budgets for a single fiscal year. You have to translate the study budget into the corporate budget, and you have to look at actuals against the study budget and against the corporate budget. Then there's the whole reforecasting element. So understand how the solution handles reporting in all of those different ways, and how dynamic it is in comparing them.

Our white paper goes into a lot more detail on how to evaluate. But I'd throw it back to the audience: what's top of mind for you? Then I can give examples of what to dig into based on that need.

Bethany Lavery: We have plenty of time at the end for Q&A. Maybe you could elaborate on how this differs by company stage.

Jennifer Kyle: I'm going to say something unusual for a tech founder and CEO: in the very early stages, you don't need software. If you're a very small company with a couple of people, preclinical and not yet in the clinic, just use a workbook. I know CMC accruals are a pain and very time-consuming, with a lot of lines and a lot of data, but at the end of the day you can straight-line to 100 percent complete. In the early days of Condor we were selling our software as workbooks - we gave the CMC workbook away for free, because it's an easy way to automate that.

Where you might want something more robust at that early stage is when you start planning to go into the clinic. What does that cost? How much should we budget for Phase 1, and then 2 and 3? That's about the whole asset and long-range planning for it.

On the emerging side, say you're in the clinic with a couple of Phase 1 trials. They're safety trials in healthy volunteers, 10 or 20 patients, one site or a few at most, usually under 12 months. You can do that in Excel pretty easily, and even if it runs long it isn't that complicated.

That said, as you move into, say, an oncology study going from Phase 1b into 2, you'll have a lot of protocol amendments, a lot of dose escalations, and a lot of change orders. That's when you might start thinking that even though this is a small-scale trial, we know there will be a lot of complexity around the protocol, a lot of changes, and things moving fast - so let's bring in a solution that streamlines it, instead of taking on the headache ourselves of designing the reporting and the process. Get a solution out of the box, install it, and let it do the work for you.

Beyond that, once you're in the clinic with several trials, if you're not already using software, this is about the time to start thinking about it. And it goes back to the triggers. Are your Excel sheets breaking? If you have 300 tabs, it's not if it breaks, it's when.

Holly Reeves: Jen, I hope you get it figured out before you reach 300 tabs. That already seems like too much.

Jennifer Kyle: I have reviewed those. I did it once, I said this is never happening again, and then Condor was created.

Bethany Lavery: Jen, we have a question in the chat. It's probably a loaded question, but is there a ballpark number of sites or patients where the Excel process starts to break and you should start thinking about a system?

Jennifer Kyle: It's not really the number of patients where I see the complexity. It's the number of clinical sites and the complexity of your protocol. In an oncology study you may only have 200 patients, but 100 sites spread across regions, a lot of protocol amendments, and procedures like biopsies and MRIs - the invoiceables on investigator fees that are a real pain to estimate. That's a good trigger even with a small patient population.

On the flip side, once you get to 50-plus clinical sites, and certainly with mega studies of 700 sites, there's no way you should be doing that in Excel these days. Just use software.

Bethany Lavery: When companies are thinking through using software, what questions should they ask when selecting one?

Jennifer Kyle: Again, it goes back to why you're buying software, and you'll want to click into the areas the software says it can solve for you.

Number one, if accounting and accurate reporting are critical, you'll want to understand the SOC 1 - and actually read the SOC 1. Understand whether it covers everything you need it to based on your own process, and bridge the two together. I'm sure Holly can add color there.

Holly Reeves: You're stealing my thunder for the next section.

Jennifer Kyle: Go ahead. I have more questions, but we're on that thread. What do you think about the SOC 1?

Holly Reeves: It's critically important to have one. Even more important is that the SOC 1 covers not just whether the system has the right IT controls, but whether it has the right reporting. Is the reporting you want to rely on covered from a completeness and accuracy perspective, so that you put the parameters in, push a button, and have what you need without doing anything further? What are the user considerations? And does the organization have the right key controls?

You can have a system all you want, but if you don't have the right processes, procedures, and controls over what goes into it and what comes out of it, the system is only as good as what you put in and get out. Making sure those controls are in place is critical.

I always tell Jen how good it is that she was an auditor at heart while creating clinical software, because she makes sure the right SOC 1 is in place so you don't run into issues later around the information coming out. Whatever software you put in place, whether it's booking accounting entries or connected into your ERP, it produces information that goes into your financial statements. Back to my earlier point about management being responsible: management needs to make sure whatever comes out of any clinical software is complete and accurate based on all the information present, and that it's reported accurately.

So it starts with the SOC 1, because you need the IT controls and the right reporting. But it's also about any other user considerations the service auditor thought were important, and about having the right controls in place at your organization so management is comfortable with the information that ultimately makes its way into the financial statements.

Jennifer Kyle: To that point, it's about reviewing the inputs and reviewing the output. Because of my background - thanks for calling that out, Holly - we have a process. Not only is there the SOC 1, but as you go through the workflow in Condor there are review procedures. I call it clinical accruals for dummies, though I know none of us are dummies: follow these steps and you're covered. If you follow the checklist and you document, you're covered from a SOX compliance perspective to the extent that you follow it.

Another question to ask on reporting specifically is how the software handles retrospective changes. This is a biotech nuance. There are so many changes, and sometimes a change goes all the way back to the beginning of the trial and you have to true up. Sometimes a change order is effective in June and you only backdate it to June, but I don't see that as often.

So when you have a big retrospective change, you want to understand the impact, and you want to know: if I make a change, how does it roll through the software while keeping my prior period reporting intact so it doesn't mess up the numbers? If the system recalculates everything from prior periods and then uses that to populate your journal entry, you'll have an incorrect journal entry, because it's catching up things you've already booked. Understand how the system processes retrospective changes, and what controls are in place to preserve the integrity of prior reporting.

Another one, when vetting solutions, is to ask the company about their customers' experience. Are customers using it for just one trial, or across their full financial process and their whole portfolio? That tells you how much trust they have in the system. If it's a big company with 15 or 20 trials and they're using the software for one program, you may want to ask questions. Maybe there's a good reason, and that's okay, but ask, and understand why it isn't used across the portfolio. Then trust but verify: go talk to that customer and ask about their experience.

Another is to think about your own process and understand how much of the solution fits out of the box versus how much you'd need to change your process - which might be a good thing - or how much you'd need to build custom on top. The more custom you build, the more you're just replacing an Excel workbook with a system you're force-fitting. You're overcomplicating things, and that weighs on the ROI of implementing a solution.

This goes back to an earlier point. We want to get more efficient overall, so how do we rethink the R&D financial process altogether? When you're implementing a solution, software is not going to cure all. You need to combine process and solution, and when you do that you also need other key stakeholders on board.

Holly Reeves: Can I add one more, Jen? Whenever I talk to people who've seen the white paper and ask what to think about, I tell them to make sure they're doing demos. No different from any system implementation, there's a whole user acceptance testing period.

Before you make a vendor selection, make sure the system does all the things that you and other key stakeholders need - and this should be well outside finance. It should include clin ops, and if there's a CMC aspect, quality. Whoever you've identified on your steering committee as key stakeholders should be involved, and all of them should see a demo of what the system can really do. To your lovely auditor trust-but-verify statement: anyone can say anything. Make sure you see it live in the system and that it works, if it's important to you, before moving forward.

Jennifer Kyle: Good point. Sandbox. You can get a sandbox environment - my team is going to kill me for saying that, but the former consultant in me says that's what I'd be doing.

I also see on the polling question that the highest-rated consideration was resources and time for an implementation. On that note, another question to ask is to talk to references - not just the references the vendor gives you. Those are great, and they should always come out great; if they don't, that's not good. But go back-channel too.

When you back-channel, ask how the implementation process went. How long did it take? How much time did it take on your end? How many resources did it take on your end? A software vendor might tell you it takes a number of weeks and little of your time, but trust and verify - ask others about their experience. And always ask why, because there may be situations where something was required due to a nuance in their organization that isn't relevant to you.

One more thing I wouldn't forget to ask: what are the maintenance requirements post go-live? That's part of your resource and bandwidth picture. You're putting in a solution expecting it to save you time, so look at how much time it has really saved - not just from an ROI perspective, but what resources are required to maintain the software. That's both a user-level, day-to-day question and an IT question about managing the integrations or being the system administrator.

Bethany Lavery: All good points. Anything else on what to consider for vendor selection?

Holly Reeves: Let me summarize a few key points from what I've seen in the marketplace.

To start, align all stakeholders on what everyone expects out of clinical software. Those stakeholders could include IT, finance, clinical, and accounting, and you want the right steering committee in place to make this a successful process and implementation, no different from any system implementation.

Define your success metrics upfront. And to Jen's point on reference checks, find out what the historical pitfalls were and have a plan to avoid them. Be clear on exactly what the entire organization needs out of the system, and set those expectations early.

Once you think you've made a decision or have a pathway forward, get alignment and buy-in from senior leadership. Make sure they understand the resources involved - both the cost, since it's an interesting market right now and money can be tight, and how much of people's time it will take. You don't want to get far down a path and realize the executives or the board aren't aligned on the time or the cost, and you've wasted a lot of effort.

Lastly, the point I made before: make sure you've got the right demos and use cases, that you've actually seen in the system what you're expecting and what you aligned on with key stakeholders. It really is trust but verify - confirm the product will work for the organization and for what people are looking for.

Jennifer Kyle: I want to add to something you said, Holly, that a lot of folks on this call should consider: getting cross-functional buy-in, and understanding from your CFO whether you have budget for this or how you make budget for it. You don't want to go through the whole process only to hit a roadblock when clin ops says, we're not giving you our EDC data. We lived through that situation. We always end up getting it, but get that buy-in first.

Socialize it. Get out of your seat and go talk to those stakeholders and understand their pain points. Come at it from a consultative side. Ask clin ops: how is your experience when we're meeting to go through audit confirmations? How is it when you're reviewing invoices from the CRO? Would you like to save some time here? If they're open to it, you can have more of that dialogue, and then you have a champion. Everybody in finance and accounting knows that if you can get your clin ops team on board, you can basically get your way with anything.

So instead of treating them as the other side of the house, partner with them. Ask how you can add value to their workload, and then bring that to your CFO. Then you have a win-win for the organization, and some great bullets for your resume at the next IPO.

Bethany Lavery: Thanks. One other topic for Holly, and I know you touched on some of this, but we know it's your favorite. Once the system is selected and in place, how do the auditors get comfortable with all of it?

Holly Reeves: Great question. Let me start with what controls I'd expect management to have in place before the auditors get started.

I talked about the SOC 1, but let's talk about how you go from Excel, or whatever your tracking system is, into the new software. You'll have a development and conversion process. Step one is how management gets comfortable with the information that becomes the starting point in the system, and what controls and procedures management performed to make sure the information housed in the ERP or in Excel files was completely and accurately entered - and what documentation management has to evidence that review.

Then, as auditors, we'll test both completeness, that all the information got entered, and accuracy of the information that's in the system. If you don't have the right starting point, you'll have a problem later.

The SOC 1 generally covers a lot of the IT and system information, but as there's change management, new reports, or customized reporting, management needs the right controls over that as well. It comes back to completeness and accuracy: how can management prove it out?

Ultimately it's about the organization having the right controls and checks so they have the right numbers in the balance sheet, the P&L, and the footnotes, and so the auditors can design and perform their procedures to support what has come out of the system. To my earlier point: garbage in, garbage out. There are processes, procedures, and controls for getting information into the system, getting it out, and flowing it through the financial statements, and management should get those to their auditors early and upfront to make it easier on the back end.

Bethany Lavery: Jen, Holly, we're running out of time, and I want to thank you both for taking us through how biotechs can transform their finance operations. Let's open the floor to questions from the audience, along with a final polling question.

Jennifer Kyle: I see a question. Are we talking about CROs or CMC? Mickey, we're talking about both, but CROs are really the big ones - the ones that get really complicated. Those are the ones where you want a good way of coming up with your own independent estimate by leveraging clinical data, rather than just trusting what they give you on day five or right before you file your 10-Q.

Next: how is AI being used in accounting, not finance - day-to-day accounting, reporting, or other areas? AI is a hot topic right now. I'll speak to Condor rather than to others in the industry or to what's theoretically possible.

For accounting, number one, you're always going to need somebody to review the numbers. If AI spits out a number and you book it without doing anything, AI could be wrong. You want controls over what you're looking at.

At Condor we think about AI in two ways. One is data ingestion: mass uploading contracts and using that information to auto-populate fields. Even there, you still need somebody to review it and confirm every field was input correctly and tagged the appropriate way.

The second one I'm particularly excited about is only possible because of large language models. For Condor, it's a model at the intersection of the clinical data and the financial information. Think about flux analysis: if you want to understand why your actuals changed against budget, Condor knows how the actuals were calculated and how the budget was calculated, and we have the data to support whether the change was driven by patients, sites, or time. That, coupled with a large language model, can do flux analysis that off-the-shelf models can't, because it's built on that specific clinical data set. That's not live right now, by the way, but it's what we're building at Condor specifically for accounting.

Christopher asked: what are the biggest risks in transitioning from Excel spreadsheets to a software solution, and how does Condor address them? I'll answer, and then Holly, you can add what you're seeing on risks in software generally.

The biggest one is this. If you were doing something in Excel and the software now produces very different numbers for prior periods - even just the prior quarter - then either what you were doing before was inaccurate, or how you're setting it up in the system is inaccurate. So the biggest risk is really about running the two in parallel and making sure you understand and can explain the differences. That change in estimate analysis is still required.

For Condor specifically, that's part of our implementation. For every customer we onboard, we do the change in estimate analysis for them, partner with them, and help explain the differences. It's also a quality check to confirm we set up the drivers correctly.

Holly Reeves: You hit it dead on with running parallel for a couple of quarters. No different from most system implementations, there's usually a parallel process to make sure everything matches up.

Part of the question was about timing. Excel files can be fit for purpose for a period, depending on how simple the preclinical or clinical trial is. It's about striking the right balance: when does it become too many sites, too complicated, too much to keep people on an Excel file, so that moving to a system makes sense? There isn't a one-size-fits-all answer - it's very facts-and-circumstances based. Hopefully the white paper and today's discussion help with the things your organization should be considering and asking with all key stakeholders, to decide when the right time is and how to implement appropriately and seamlessly.

Jennifer Kyle: On that, as you evaluate software and go through the process, have someone in your organization project managing the evaluation. That project manager should also be talking to your auditors: as we're evaluating, what's important to you, and how do we de-risk this for you as well? Talk to all the different stakeholders, because things will come up during implementation. There are things you don't know, and then you've implemented something and missed a key consideration from someone - an auditor, or someone on your clinical team - that was actually critical to your process. I've seen that happen. So gather information across your organization, and run your change in estimate analysis. Between those two things, you should be in good shape to de-risk whatever path you go down.

Holly Reeves: We're at the top of the hour. I've enjoyed the discussion with you, Jen, and with you, Bethany. Thank you to everybody for tuning in. This session is CPE eligible and we'll make sure everyone gets their CPE, and we'll get the recording posted so you can view it. Thank you, everybody.

Jennifer Kyle: Thanks for joining us, everyone.