Performing medical transcription analysis with Amazon Transcribe Medical and Amazon Comprehend Medical

December 2020 Update – This blog post now also covers how the Medical Transcription Analysis can also be used to store and retrieve medical transcriptions and relevant information using Amazon DynamoDB and Amazon S3 and how all of this data can be analyzed using Amazon Athena.

The healthcare industry is a highly regulated and complex industry where a great deal of information exchange occurs via verbal communication. This audio data can contain valuable information and actionable insights. This post explores how to integrate HIPAA-eligible AWS AI services Amazon Transcribe Medical and Amazon Comprehend Medical to store and identify insights in this data. Automating medical data extraction and comprehension helps healthcare professionals focus on patient care.

Amazon Transcribe Medical

Amazon Transcribe Medical is a machine learning (ML) service that makes it easy to quickly create accurate transcriptions from medical consultations between patients and physicians. Amazon Transcribe Medical automatically converts the medical and pharmacological terms used in physician-dictated notes, practitioner and patient consultations, and tele-medicine from speech to text for use in clinical documentation applications. For more information, see What is Amazon Transcribe Medical?

Amazon Comprehend Medical

Amazon Comprehend Medical is a natural language processing service that makes it easy to use ML to extract relevant medical information from unstructured text. You can quickly and accurately gather information (such as medical condition, medication, dosage, strength, and frequency), from a variety of sources (like doctors’ notes, clinical trial reports, and patient health records). Amazon Comprehend Medical can also link the detected information to medical ontologies such as ICD-10-CM or RxNorm so downstream healthcare applications can use it easily. For more information, see What is Amazon Comprehend Medical?

Medical Transcription Analysis

Medical Transcription Analysis (MTA) is a simple solution that uses Amazon Transcribe Medical and Amazon Comprehend Medical to provide medical notes transcription and comprehension. The solution opens a WebSocket between the client (browser) and Amazon Transcribe Medical. You use this WebSocket to send the audio from the client to Amazon Transcribe Medical and retrieve real-time transcription, which is then rendered on the UI. The transcribed results are then sent to Amazon Comprehend Medical, which returns an analysis of the transcription. The following diagram illustrates this architecture.

Deploying MTA

For instructions on setting up MTA, see Medical Transcription Analysis on GitHub.

The deployment creates an Amazon Simple Storage Service (Amazon S3) and Amazon CloudFront backed website with authentication provided by Amazon Cognito. It also creates an AWS Identity and Access Management (IAM) role with permissions to Amazon Comprehend Medical and Amazon Transcribe Medical, and an API to retrieve temporary credentials from the role. The deployment also includes Amazon DynamoDB tablesan Amazon S3 bucket for data storage and Amazon Athena for data analytics. 

Using MTA

After you deploy the application, you receive an email with login credentials. When you log in, you are directed to the homepage. The homepage offers options to dictate audio using the microphone, upload a sample audio file or play a sample audio files. The following screenshot shows the MTA homepage.

If you choose a sample audio file, MTA opens a WebSocket with Amazon Transcribe Medical and renders real-time transcription results on the UI. You can highlight words that fall into different medical categories. The following screenshot shows a transcription from the sample audio.

When the transcription is complete, you can choose to Save Session or Analyze. To Save Session, you provide a session name and tag a health care professional & patient (or create them) for the session. These saved sessions can also be searched and retrieved for future use.

Choose Analyze to get the identified medical terms associated with medical categories such as Protected Health Information (PHI), Medical Condition, Anatomy, Medication, and Tests, Treatments, & Procedures.

Under certain categories, you can also find popular terminologies and codes associated with the highlighted words. For example, in the Medical Condition section, you can find ICD-10 CM concepts, and the Medication section contains related RX-Norm concepts. The following screenshot shows the analysis view of the Medical Condition category. You have the option of hiding any of the entities by clicking the blue eye icon. Hidden entities will not be recorded in the summary section for this session.

To export this information, choose Summarize. On the Summarize page, you can find all the results extracted and identified from the audio track. You can export this information as a PDF, which enables your downstream dependencies to consume this data. The following screenshot shows the extracted information from the earlier transcription.

As mentioned above, sessions can be saved for future use. To search for these saved sessions, click on Search in the navigation bar. You can then search the sessions either by  inputting the health care professional or patient ids with the option to provide the session id.

Results matching the search criteria are displayed in a tabular format. You can click on these results to view a summary of the session

These saved sessions are stored and retrieved from an S3 bucket. You can also analyze the saved sessions via Amazon Athena. The MTA deployment pre builds a basic catalog and queries for common analytical questions. You can update the data schema and saved queries for your business use case.

Optionally, you can also use Amazon Quicksight with these Athena tables for dashboarding and reporting.

Summary

This post reviewed how you can integrate Amazon Transcribe Medical and Amazon Comprehend Medical to transcribe audio data, extract key medical components, and tag the data to their corresponding entities. Automating the medical transcription and comprehension process makes it easier for healthcare professionals to focus on patient care. This integration also processes the results into easily digestible formats, which reduces the manual effort needed to record and digitize information.

To access the MTA source code, see Medical Transcription Analysis on GitHub .This solution has been made open source so that AWS customers can extend and incorporate the solution into their workflows. Possible extensions include integrating into EHR systems, adding a persistent storage layer, building analytics over collected data, enabling batch processing and enhancing the user experience for multi-speaker / conversational use cases.

About the Authors

Simran Baxendale is a Program Manager in the Amazon Machine Learning Solutions Lab. She helps define and coordinate program strategy for the Demos team.

Alex Chirayath is an SDE in the Amazon Machine Learning Solutions Lab. He helps customers adopt AWS AI services by building solutions to address common business problems.

Shivani Mehendarge is an SDE in the Amazon Machine Learning Solutions Lab. She helps create demos that encourage clients to integrate onto the AWS AI platform.