HIT Consultant – Read More

Artificial intelligence in healthcare is moving into a new phase. What was once dominated by research pilots and one-off experiments is now becoming a matter of ownership. Hospitals, medical device firms, insurers, and digital health startups are no longer just testing algorithms. They are building, buying, and maintaining AI products with real accountability. This shift has significant consequences for regulation and for patient outcomes, and it raises questions about how the healthcare system adapts to a landscape where algorithms are as much products as medical devices.

When AI becomes a product, the role of the product owner becomes central. In agile or scrum teams, product owners sequence features, prioritize backlogs, handle tradeoffs, and align development with business objectives. In larger organizations, there may be multiple product owners specializing in data products, business products, or technical products. In startups, however, the product owner often wears multiple hats, acting as product manager, business analyst, or even scrum master. I have personally worked in both these settings and have seen how product ownership can vary significantly, with unique challenges for each based on stakeholders, regulation, scope, and dollar investments involved.

For AI in healthcare, product owners face unique challenges. They must ensure clean datasets for training, manage vendor selections, balance commercial and clinical metrics, and constantly iterate on performance. Beyond functionality, they are accountable for how the product integrates into clinical workflows, whether it scales, and how it adapts as data evolves.

What I have seen in my career while working with both operational and clinical stakeholders is that errors in sequencing features when you are implementing a new EHR, EPM, or ERP system in healthcare can have direct clinical and patient outcome consequences. In comparison to consumer AI products, healthcare AI products must also adhere to various regulatory and governance standards like HIPAA, FDA clearance, etc,. and ensure that the iterations in the models don’t break or overlook any of those standards. So, it’s important for product owners to balance the demand of the business while still ensuring safety, validation, and compliance checks early on in discovery, requirement elicitation, and backlog refinement sessions.

AI in Medical Devices and Regulation

Unlike drug R&D, which often relies on AI in early discovery, medical devices present a more immediate set of regulatory challenges. The FDA has issued guidance on software as a medical device, including AI-based tools. The challenge is that, unlike a pill or implant, an AI product may evolve after it is deployed. Regulators must decide whether updates require re-approval, how to evaluate ongoing safety, and what transparency standards should apply.

Examples already illustrate these tensions. In 2021, the FDA cleared IDx-DR, an AI tool that autonomously detects diabetic retinopathy. It was groundbreaking but also exposed gaps in defining accountability when AI acts independently. More recently, Epic integrated generative AI into its EHR system to draft clinical notes, prompting hospitals to form internal oversight boards since regulatory frameworks are still emerging. While this is an exciting integration for sure, especially considering the volumes in busy care-site settings, it still begs the question of how much trust physicians or clinicians in inpatient or outpatient settings can have in the quality of the EHR AI outputs, considering patient outcomes are at stake. 

We all see that AI-enabled medical devices evolve continuously. It’s more important than ever for Medtech organizations to establish separate teams to ensure regulatory readiness of the products they are building. They can have compliance and product teams work together to ensure both regulatory and quality adherence. From my experience, I have seen successful teams enable formal change control processes, as well as know when to and how to pivot if things do not go as planned. This can help Medtech organizations have an additional shield against regulation and auditing authorities.

Balancing Innovation, Equity, and Accountability

AI tools have the potential to accelerate diagnosis, personalize treatment, and make care more accessible. But opaque algorithms can also reinforce inequities, as seen in risk scoring systems that underestimate the health needs of Black patients due to biased cost data. When these models scale as products, their impact multiplies.

That is why governance and ownership structures are critical. Product owners, compliance teams, and regulators must work together to ensure fairness and transparency without stifling innovation. Hospitals are already setting up AI governance boards to evaluate products before adoption and monitor their performance after deployment.

Product Owners can establish sustainability for AI features by prioritizing those alongside managing clinical risk. There can be frameworks established where features involving clinical outcomes face tighter scrutiny before approval, while those for enhancing operational workflow can have lighter checks in place. Transparency with clinical stakeholders to explain how algorithms work, and what datasets are being used  is important to gain trust while also ensuring abstraction in conversations so that the stakeholders are not lost in technical details.

Today, big tech firms are setting standards and startups are driving innovation in the clinical and Medtech AI space while strategically partnering with Healthcare and Pharma giants. To ensure scalability in this area, these firms must embed practices and frameworks from regulated industries because success depends not just on AI innovation but on balancing it with governance, compliance guardrails, and accountability. For product leaders, it means entering a new phase of innovation in the product lifecycle, embracing uncertainty, and treating AI as a living and evolving product, not just another feature. 

About Akram Hossain

Md Akram Hossain is an accomplished business analysis and product management leader with over nine years of experience leading enterprise-wide digital implementations across healthcare and technology. An MBA, Advanced Certified Scrum Product Owner, and Certified Business Analysis Professional, Akram specializes in EPM, ERP, and EHR systems, with deep expertise in platforms like Anaplan, Workday, SAP, and Epic. Akram has a proven track record of defining product strategy and roadmaps using design thinking concepts, optimizing workflows, and driving operational efficiency through Agile and Scrum methodologies.