BRIGHTCOVE AI METADA ENHANCER

Overview

Brightcove is a leading video hosting and streaming platform, helping businesses deliver engaging, high-quality video content at scale. Their services include video hosting, live streaming, content monetization, and analytics. With solutions tailored for various industries, they assist clients in enhancing customer engagement, boosting revenue, and optimizing video performance through advanced technology and AI-driven tools.

I was assigned to the AI Metadata Enhancer project to integrate AI suggestions for metadata into the existing user flows and user interface of Videocloud.

Role

Led the design of this project from research to implementation.

Challenge

Integrate AI suggestions for metadata (titles, tags and descriptions) into the existing Videocloud UI in an intuitive, engaging and seamless way.

Collaboration

Product Manager
Technical Lead
Software Engineer

PROJECT DETAILS

Background
The AI Metadata Enhancer is the first feature based on GenAI approved for production, setting the first milestone of the AI adoption journey. The AI metadata enhancer represents our strategic focus on using the most advanced technology to continuously improve and make our customers workflow more efficient.

Problem
According to the last analysis, more than 50% of our customers’ videos do not have tags, custom fields or relevant metadata, limiting their capabilities of discoverability, targeting and meaningful analytics aggregations.

Creating meaningful metadata becomes an operational burden for customers uploading multiple videos in a day or “one man” teams dealing with multiple tasks at the time, making metadata generation a low priority one without realizing the impact of it. Metadata enhancer aims to be a time saver, the path to more views and better decision making for these use cases and anyone looking to streamline their publishing workflow.

Solution
System implementing an LLM model capable of processing an asset’s transcript or caption file and generating meaningful metadata oriented to describe the content, optimize discoverability, improve advertising targeting and generate better insights.

Success metrics

Adoption: 30% of the new videos ingested in accounts with this featured enable should request metadata suggestions after 3 months of GA.
Engagement: 50% of customers buying the AI Content Suite should use Metadata Optimizer at least once per week on average. This number should increase to an average of 3 times per week after 3 months of use.
Accuracy: At least one of the metadata items suggested should be accepted 60% of the times.
Effectiveness: Assets with metadata suggestions should score an increase of 20% in play rate than the ones without a suggestion.
Market Competitiveness: RFPs and opportunities requesting AI features should be satisfied by the Metadata Enhancer first version and its roadmap.

Design kickoff

Workshop
To start designing for this project I conducted a workshop with the design manager, the product manager of the project and two software developers.

The goals for the workshop were to define the guiding question, user persona and user flow. And also to start ideating on some of the design requirements needed to provide a seamless and intuitive experience.

Guiding question
How might we seamlessly integrate AI-powered metadata suggestions (titles, tags, and descriptions) into the Videocloud UI, ensuring an intuitive and engaging user experience?

User persona
Meet William. William is a Video Publisher

Goals
• Create more engaging content and increase viewership.
• Boost retention/traffic/sales.
• Speak for the company.
‍
Motivations
• Be fast and efficient while publishing videos.
• Recognition, be known as a creative copywriter.
• Generate leads, increase retention/traffic/sales.

Tasks
• Identify and fill out the metadata of videos (title, short and long descriptions, tags, etc).
• Gather reports and analytics, measure impact and iterate.
‍
Pain points
• Large amount of data to consider.
• Time consuming tasks.
• The quantity of videos published a day can be huge.
• It is a repetitive task.
• Uncertainty on how to tag my videos based on the market.
• It can be hard to come up with ideas for the metadata of the video.

User flow

Questions to help build up the experience

How do we display the suggestions?
Do we want to make a comparison?
We should provide feedback so the user knows something happened, how do we do so?
How can the user navigate through the options? How can they generate more suggestions?
Can the user save individual fields? How do we allow this?
How can we allow the user to go back to what they initially had?
Do we want the user to be able to edit the AI suggestions?

Innovation sessions

The Brightcove Innovation Program allows Brightcove customers to participate in product research, and give feedback on new features and functionality.

Some of the benefits are:

Get customers feedback through different points in the product life cycle
Insight & confidence in building PRDs for use cases/pain points
PM & Design build direct relationships with customers
Identify audiences for EA to features and functionality
Customers have an opportunity to be heard
Drives customers to the community

We ran five innovation sessions with different clients.

Clients

Consumer Reports
Dell
Dallas Black Dance Theatre
Ultimate Kronos Group
Newscorp Australia
SAS Institute
Stuff NZ

General topics
The results from the innovation session could be divided into the following blocks:

Quick discovery
Current experience
Pain points
Improvements
Concerns

Key takeaways

Usually entering metadata in another platform, would need to modify workflow a little to take advantage of GenAI metadata from BC. (Session 1)
Would like transparency about the AI, is it developed by BC?, is it a 3rd party? How is it learning? where is it pulling the information from? (Session 1)
It seems like this set of customers could benefit from the work that is being done to get metadata from AI. (Title, Description, Tags and Thumbnail). (Session 2)
Chapterization is a common topic among our customers, they would like to have the AI help with it. (Session 2)
Current approach to GenAI metadata could be useful for this client. The fields they would like to have help with are the ones we are currently working on. (Session 3)
Consistency with titles, descriptions and keywords is one of the major pain points since they have a bunch of producers and each producer has its own way to categorize the content. (They usually train the producers to help with this). (Session 3)
In their content they prioritize quality over quantity, and they would like the suggestion from GenAI to also be good quality. (Session 3)
The current work that is being done to get metadata from AI seems like it could add value to this client. (Title, Description, Tags and Thumbnail). (Session 4)
By what they said about the BCOV’s transcriptions not being very reliable, if we rely on our users/clients using it, the AI generated metadata might not be as accurate or useful. (Session 4)
The current scope of the GenAI metadata project would be helpful for this client to save time, have more consistency and be more accurate. (Session 5)
Concerns about AI generating Maori content. (Session 5)
Nothing goes out of the cloud. AI is not being trained on their data. (General concern).

EARLY ADOPTER PROGRAM

After the design was done the AI Metadata Enhancer was released in a kind of a beta stage, for a set of customers that signed to be part of the early adopter program, to be the first one to try it out and provide feedback.

Realizations
During conversations with customers we observed two more key moments where users think about curating the metadata:

At Ingest: Set metadata while uploading the content and take advantage of that wait time
Bulk Adding: Organize recently added content in bulk in the media module after ingest was completed.

User stories
As a Studio user I want to be able to process multiple videos with Metadata Enhancer and come back to review the suggestions once the process is completed so I can feel like I’m being productive and efficient.

As a Studio user I want to be able to upload a video and have Brightcove transcode and suggest metadata as part of the initial processing and later review the video details along with the metadata suggestions so I can feel like I’m being productive and efficient.

Implementation
At Ingestion Processing
Add controls to the Upload panel to allow the customer to select if they would like to have the Metadata Enhancer generate suggestions for the asset. The system will update the user of the status of the process at all time and notify them when the suggestions are ready to be reviewed.

This option will require auto captions to be processed at ingestion.

Bulk Processing
Add to the Media module the capability of processing videos in bulk with the AI Metadata Enhancer to make their operation more efficient and reduce the time spent on going through the processing activation and wait time.

The Media module will offer an easy way of selecting videos, triggering the processing and accepting the highest ranked suggestion.

The user will be notified when the processing has been completed for them to review and approve. The Media module should show at a glance that the metadata suggestions have been generated and that they are ready for approval or to be reviewed in the video details page.

This option should prompt the user with the videos that do not have a text track to be processed and offer the possibility of triggering the auto caption workflow as part of the processing.

Status Widget
Take up again the usage of the status widget and adapt to be able to display when the processing of the AI Metadata Enhancer has been completed and is ready to review.

PAIR Projects Impact

At the same time that the AI Metadata Enhancer was being worked on. Other projects were also evolving and significantly impacting and touching some of the screens that were updated for this project.
‍
The design team detected some inconsistencies between the same screens for different projects and also some inconsistencies between the same components for different projects and decided to act on it and work on the consistency for all projects within the platform.

Video details header and CTAs
With so many projects and integrations going on, we detected an inconsistency in the video details header and call to actions.

It is also important to mention that with so many projects related to AI and other features the quantity of CTAs was becoming unmanageable.

Here are some images on how it was looking for different projects:

After a few meetings to brainstorm and ideate on the best way to organize and group the CTAs this was the final proposal:

Bulk-in-action bar
Once the CTAs in the video details page were updated and regrouped, we thought it would make more sense to also update and group the actions in the same way we did but now for the bulk-in-action bar.

This is how it looked before:

And this is after the update:

Status widget
For the status widget we needed to adapt it to support the display of the processing of all the different projects.

It ended up like this:

The default view is a list ordered by latest update, but it can be changed to be by group and/or oldest update.

More descriptive labels
We decided to change the labels for more descriptive ones, we went from Generate with AI to Generate Metadata and Generate Images.

DESIGN FOR GENERAL AVAILABILITY

NEXT STEPS

Measure success
After the General Availability (GA) release, it's crucial to focus on measuring success to ensure the adoption and engagement. As UX designers, we play an integral role in monitoring and improving these metrics. Specifically, we will assess adaptability and engagement through user data and behavior tracking in tools like Gainsight, leveraging heatmaps, usage patterns, and other relevant metrics to optimize the user experience.
Innovation sessions
To foster ongoing product innovation and ensure that we are consistently enhancing the user experience, it is essential to conduct another round of Innovation Sessions. These sessions will serve as a valuable opportunity to gather real-time feedback, evaluate the latest features, and explore new ideas that can contribute to product improvements.
AI mark
When we ran the feedback sessions with customers after the early adopter program was released, we learned that it will be useful and helpful for users to identify which videos metadata was filled with the AI Metadata Enhancer from the table view and also from the video details view. Due to time and resources it was not possible to implement it for the GA release but it is in scope for the next release since this is something that will improve our users flow by speeding up the identification of the videos that haven't apply suggestion from AI and prevent users from using AI more than once (if it is not desired) in the same video.
Integrations ingestion flow
One of the features/flows that were not in scope for the first release was enabling to get AI Metadata suggesting when ingesting a video from the integrations offered (Dropbox and Google Drive). So for future versions this probably will get on scope and be released in the upcoming releases.

Integrating the AI Metadata Enhancer

‍NDA ⚠️
In compliance with the non-disclosure agreement I signed, this is an overview of the work I'd done at Brightcove. I have omitted sensitive data so no piece of information is compromised.

BRIGHTCOVE AI METADA ENHANCER

AI Thumbnails

Obtain high-quality video thumbnails.

AI Chapters

Generate a list of chapters (titles and timestamps).

AI Content Multiplier

Detect key moments and generate short clips out of long format content.

Integrating the AI Metadata Enhancer

‍NDA ⚠️In compliance with the non-disclosure agreement I signed, this is an overview of the work I'd done at Brightcove. I have omitted sensitive data so no piece of information is compromised.

BRIGHTCOVE AI METADA ENHANCER

AI Thumbnails

Obtain high-quality video thumbnails.

AI Chapters

Generate a list of chapters (titles and timestamps).

AI Content Multiplier

Detect key moments and generate short clips out of long format content.

‍NDA ⚠️
In compliance with the non-disclosure agreement I signed, this is an overview of the work I'd done at Brightcove. I have omitted sensitive data so no piece of information is compromised.