Not a day goes by that we don’t mourn the loss of the epic mobile streaming app known as Quibi, short for “quick bites.” 

Just kidding. We were today years old when we learned Quibi ever existed. We only found out about it because we asked Google to bring us a sampling of the biggest digital product failures ever. 

Ouch.

Quibi (RIP) launched in the newborn days of COVID-19 shutdowns: April 2020. It was supposed to be the newer, better Netflix, but for your phone only. That’s right, no switching between your laptop, your TV, or your phone as you see fit. 

At a time when streaming services were suddenly necessary to help keep the anxiety and stress of the pandemic at bay – Netflix racked up 15.7 million new users in the first three months of 2020 – Quibi should’ve been a hit. 

It wasn’t. 

Within six months, Quibi had spent the more than $1.75 billion it had raised from investors, and it shuttered its doors. 

What went wrong?

A lot, obviously. But we’re here to argue that if Quibi’s founders had paid more attention to product testing and what potential customers actually wanted from the streamer, they could have succeeded – or at least avoided flushing over a billion bucks down the drain. 

What is product testing?

Product testing is showing a product concept or prototype to your target market to gather feedback on whether they actually like it. It’s a way to test whether the product is viable before investing billions of dollars into full-scale product development. 

Let’s go back to our friend Quibi. Founded by former Disney president, Jeffrey Katzenberg, and led by CEO Meg Whitman, formerly of eBay and Hewlett-Packard, the idea generated a ton of buzz among investors – most of whom were Hollywood giants like The Walt Disney Company, 21st Century Fox, and Sony Pictures. 

With so much backing, you’d think Katzenberg and Whitman would thoroughly test their product idea to determine market fit before the official launch. 

Wrong. 

As the product was dying and Katzenberg and Whitman were shutting things down, Katzenberg said in an interview, “We asked people to pay for [Quibi] before they actually understood what it was.” 

Obviously, we don’t have access to Quibi’s internal information. We don’t know if they rigorously tested their product at every iteration to make sure target customers wanted or needed such a platform. 

But Katzenberg’s quote makes us doubt they did much product concept testing at all, much less beta testing throughout the development process. Because if they had, people would have understood what Quibi was and why they needed it. 

Product concept testing helps avoid the deadly false consensus effect

Some Quibi critics say Whitman and Katzenberg fell prey to the false consensus effect.

The false consensus effect is a cognitive bias where you assume that since you think and act a certain way, you assume lots of other people think and act that same way too… even if they don’t.

So basically, since Katzenberg and Whitman both thought 10-minute videos you could only watch on your mobile phone were a great idea, they assumed tons of other people would think so, too. 

They didn’t.

And assumptions like that are the killers of startups and new products everywhere. 

When you’re excited about a new product idea, it’s tempting to avoid gathering feedback via A/B testing, focus groups, and other testing methods. 

Feedback means critiques, and if you pay attention to critiques, you’ll probably have to kill some darlings. Or at least re-iterate them to fit user needs. 

Not everyone is willing to do that – even if it’s the difference between success and failure. 

In Quibi’s case, the streamer depended on three selling points: 

• Celebrities. Lots and lots and lots of celebrities. Judge Judy but with Chrissy Teigen, anyone? How about Liam Hemsworth playing a dying dad-to-be who must participate in a knockoff Hunger Games to secure his wife and baby’s future? If that’s not your style, want to watch J. Lo and Kristen Bell hand out $100,000 to randos while you struggle to pay your bills? No? 
• Short-form videos. Since most subway rides (if you’re lucky) and lunch breaks (if you’re not lucky) are 7-10 minutes long, short-form content will be a hit, right? It’s not like we already have TikTok or YouTube or anything that can fill that gap…
• Turnstile-format watching. Cool, you can watch in full portrait mode if you want! Hopefully this makes up for the fact that you can only watch Quibi on your phone. Not on your TV. Or your laptop. Or even your tablet. JUST YOUR PHONE. 

via GIPHY

The harsh truth is this: no one wanted or needed these features. 

A few rounds of product testing would’ve told Quibi’s leaders so. They could have scrapped the product idea right then and there, saving literally billions of dollars. Or, they could have modified it, running proof of concept testing until they found an idea the target market needed (or wanted).  

They could’ve created a minimum viable product (MVP) and focused on refining functionality and design in response to user feedback.

Quibi could have been something that flew off the shelves of the App Store and Google Play.

Since you probably don’t want to fall into the same trap as poor Quibi, here’s how you can use product testing to your advantage. 

This process applies to digital products like our real-world guinea pig for this article (Quibi) as well as physical products. There are dozens of real scenarios we could’ve plucked from the internet to show you. (Twitter Peek, anyone?)

But the bottom line is this: product testing matters. It’s critical to a successful product launch. 

Here’s how you do it. 

Testing strategies for a successful product launch

The first step in any product development strategy is to validate your concept. In broad terms, this means making sure there’s an appetite for your product. (Our guide to idea validation can help you get started with this.)

But once you know there’s interest, you need to find out the finer details. 

Like which features are important to your target audience. Whether you’re building an app or a new type of flower vase, knowing which features to prioritize will guide your product design. You also need to find out more boring, technical things, like how much your end users are willing to pay for the product.

PickFu can help with all of this.

Our survey software puts you right in front of the most important product testers in the world: your target audience. (Here’s what PickFu is and how it works, if you don’t already know or need a refresher.)

Entrepreneurs use PickFu to test product viability all the time. In one ranked poll, a PickFu user asked which of six features is more important in a handheld showerhead: easy-to-clean nozzles, high-quality materials, tool-free DIY installation, high-pressure spray, or six-spray function. 

The six-spray function feature got the most votes, with high-pressure spray and tool-free DIY installation coming in second and third, respectively.

If we were the ones building this showerhead, we’d figure out a way to make sure those three top features all ended up in the final product. 

Why? 

Because according to the target audience, they’re important.

But validating your product concept is just the beginning. To avoid a Quibi situation, you have to bake a testing plan into your product development process. 

We’ll focus on two subcategories of product testing to help you build a roadmap: technological and exploratory testing.

Technological testing

Technological testing starts with hands-on prototype testing. As your product blossoms from prototype to MVP and beyond, you’ll keep performing these tests to make sure everything works like it’s supposed to.

There are several different types of technological tests and methodologies. They include: 

• Unit testing: This testing phase checks that the individual parts of a product (like each feature in an app) work well on their own. Isolating and testing these features solo helps you catch small bugs early on.
• Regression testing: After updates or changes, regression testing makes sure those tweaks don’t mess up existing features that were already working just fine. Regression testing is especially important for app developers because apps should get updated about once a month. You definitely don’t want to introduce new issues each time you fix an old one. 
• Stress testing: How well does your product or app perform under stress? For an app, this might look like high usage and a massive inflow of data. Does your app crash, or does it hold its own? Only stress testing can tell you. 
• Security testing: Security is a must for any digital product, including apps. And whether you’re hoping to reach a U.S.-based or global audience, there are privacy and security rules to follow. Security testing makes sure everything’s locked down tight before launch. 
• Integration testing: For app and B2B/SaaS developers, this step is critical to the product’s success. The last thing you want is for users to have trouble integrating your app with the tools you’ve promised it’ll integrate with. So make sure you test how well your app plays with each one of its integrations. 

There are all sorts of tools out there to help you set up the ideal testing environment for technological tests. 

We’ve gathered up some recommendations to help you get started:

• JUnit: This is a popular tool for unit testing in Java-based applications. It helps developers test individual components of their code to make sure each part works like it should.
• Selenium: Ideal for regression testing, Selenium is a popular framework for automating tests in web applications. It supports multiple browsers and platforms.
• Apache JMeter: This tool helps developers simulate heavy loads on applications, servers, or networks to see how they perform under pressure. It’s a solid choice for stress testing.
• ZAP: This tool is excellent for security testing. ZAP (aka Zed Attack Proxy) helps identify vulnerabilities in web applications so you can fix them before they sting your reputation. 
• Postman: Postman is a top choice for integration testing. It allows you to test API endpoints and make sure all the services within an app communicate well.

These tools can help you and your testing team pinpoint glitches in the technology side of your app. They’re for the nuts-and-bolts parts of product testing.

You aren’t done testing yet! Exploratory testing is just as important as the technological stuff. It’s what gives you the valuable, human insights you need to build a fireproof product. 

Exploratory testing

While technological testing gives you concrete, objective results (things either work or they don’t) exploratory testing is more fluid. It reveals something that’s absolutely crucial to your success: customer feedback – before you even have real customers. 

Exploratory testing is usually done in the early stages of building and developing an app. It’s also smart to do it each time you’re planning an update or revamp of your product. 

There are two main types of exploratory testing: 

• User testing: With user testing, you invite real users to engage with your product to test functionality, ease of use, and the overall user experience. Check out our guide to quick and easy user testing for more details on this type of product testing. 
• Usability testing: Usability tests measure how well a product lives up to expectations before you launch it into the world. Learn about the different types of usability testing to get started. 

PickFu is an essential tool for both user and usability testing. Even better, it doesn’t take a ton of time, energy, and planning to use PickFu to help with product optimization. We’re a rapid research tool that gets you feedback quickly – usually within hours. (Learn more about where you can post a survey.)

You can show different designs, workflows, or even specific app interactions to an audience with your target demographics. 

For example, if you’re deciding between two app interfaces, a PickFu poll can help determine which one users find more intuitive before you commit to full development. But there’s more. PickFu can also help with packaging design testing and logo testing.

In one poll, a PickFu user asked 50 respondents what features would be most important if they were purchasing a floating bookshelf meant to store records and a turntable.

The audience had some thoughts, including: 

• “I would want to know the weight limit for sure. Vinyls aren’t very heavy, but some turntables might be. Since it’s floating, I’d be really worried about putting too much weight on it.”
• “For me the most important things would be: How does it attach to the wall? How much weight can it comfortably hold? What materials are the shelf and hangers made of?”
• “The two most important things on this for me are ease of installation and stability. I want to know that I can put it up and have it stay in place once I have put things on it, especially if I had a heavier turntable and speakers.”

Weight is clearly a concern here. Of the 50 responses, many mentioned words related to the shelf’s sturdiness, including: 

• Weight (mentioned 14 times)
• Hold (mentioned 13 times)
• Fall (mentioned 5 times)
• Durable (mentioned 5 times)
• Sturdy (mentioned 4 times)
• Heavy (mentioned 4 times)

This simple usability test told our PickFu user something crucial: they have to create a product that can hold the equipment it’s meant for – without ripping the wall out in the process.

Basically, PickFu can save you from becoming the next Quibi. And our testing tools help you in the post-launch era, too – need help with your app marketing strategy, anyone? What about guidance on DIY Market research? 

From concept testing in new product development to test marketing and beyond, PickFu’s testing and app marketing tools give you the valuable insights you need to succeed. 

So if you’re thinking, “I have an idea for an app!” pause before you sprint ahead into the exciting waters of a new project. Test your product before launch with PickFu, and keep us in your back pocket for continued product management as you scale up. 

Sign up for free to find out how easy the testing process is today.

FAQs

What is product testing?

Product testing means evaluating a product’s functionality, quality, and performance before launch. It involves putting a product through various scenarios and tests to identify any issues, bugs, or improvements needed. Product testing can include both technical testing, like stress or security tests, as well as user testing, where real people interact with the product to provide feedback on usability and experience.

What is the best way to test the market for a new product?

Start by running rapid research on your target audience. You can use tools like PickFu to run online surveys and focus groups to quickly ask for feedback on product features, pricing, and branding. You can also create a minimum viable product (MVP) and release it to a small group of people to gather real-world feedback. Tracking engagement metrics and customer reactions will help you determine the product’s viability in the market.

How can I validate my business product idea before I invest money and time?

Validating your business product idea involves gathering feedback and testing demand before you fully commit. So don’t put a ring on your idea right away. Instead, run surveys or polls with your target audience to gauge interest. You can create a landing page or pre-sell the product to see if people are willing to buy it before it’s available. Running a small pilot or MVP is also a good idea. This lets you test the product with a small audience, refine it according to customer feedback, and then decide whether to move forward. 

---