Repeatability Equals Reliability: How Cradlepoint Is Automating RF Testing and Shipping Better Products

People can't stand still. Even when they're on the move, they want to be connected. Think of passengers on a bus. As they travel from place to place, they watch movies, surf the web, and use social media on their smartphones, tablets, and laptops. But how do they get online when they're on the go? It's not like the bus is trailing a fiber-optic connection to the backbone of the internet.

Cradlepoint solves the mobile connectivity problem. Our routers and cellular modems connect to cellular networks to provide mobile and branch network solutions. These mobile products also use Wi-Fi signals when possible. For example, when a bus enters a station, it switches to a Wi-Fi-based WAN connection as a way of keeping data costs down.

Our branch routers take the opposite approach. These products often are aimed at small to midsized merchants and are typically used with point-of-sale credit and debit card readers to process transactions. In this case, routers are cabled or connected by Wi-Fi to a WAN and use cellular networks as a failback to ensure that businesses can continue to make sales should their network go down. Some businesses have cut the wire completely and now use the cellular connection we provide as their primary connection to the internet.

Our IoT routers are somewhere in between. You'll typically find them in kiosks set up by independent vendors in bigger stores. These devices use cellular networks instead of the onsite local- and wide-area networks (LANs and WANs) for security reasons. Some stores have strict IT policies that forbid external contractors from logging into their networks, and a connection to a cellular network solves this problem.

Keeping Our Customers Happy

Our customers depend on Cradlepoint products for stable connections under all circumstances. To ensure that our wireless components have enough throughput to handle the needed bandwidth, we test our Wi-Fi and cellular transmitters before we launch a new product and update the firmware of existing ones. 

Shipping products that work flawlessly keeps our customers happy and reduces our tech support and engineering costs. When a product isn't performing as expected, it’s up to our engineers to determine the problem and design a solution. If we stress-test our products before we ship them, we can find any such flaws before they impact our customers.

Finding a Better RF Testing Solution

I joined Cradlepoint in the spring of 2017. I was hired for my problem-solving skills. I'm good at looking at large, complex systems, figuring out how they work, and explaining them in simple terms to other people. Our company had a Wi-Fi firmware testing process in place but needed something better. They asked me to dive into the problem and help them find a better RF testing solution.

At the time, testing our routers and access points (APs) was labor-intensive, and the results were anything but conclusive. We'd start with a test in our anechoic chamber to verify our gear's performance under perfect conditions. Next, one of our engineers would put the router on a cart and roll it around our basement to see how it performed while moving.

We would try to get performance numbers like data rate versus range, but the figures were rarely conclusive. The basement was a quiet enough Wi-Fi environment, but we shared it with other tenants, and their gear generated electronic noise that interfered with our results.

Just after I came on board, Cradlepoint started using Ixia’s IxVeriWave to test our routers and APs. It happened before Keysight acquired Ixia. IxVeriWave was—and remains—the most comprehensive RF testing solution. It was my job to figure out its complexities and how to put this new equipment to use.

Streamlining RF Testing with IxVeriWave

 A primary advantage of IxVeriWave is automation. The testing platform generates all the signals needed to check our routers and APs. It also simulates real-world conditions like crowded environments and wireless clients at various distances. We are no longer rolling carts around the basement to test our equipment and can now safely say that we've verified every wireless performance metric we can throw at our products.

Another benefit is the ability to perform side-by-side comparisons of our test results. IxVeriWave provides a stable test environment that eliminates any variables that may affect our findings. If we analyze a device on Monday, we can be confident we'll arrive at the same conclusions if we run the same tests on Friday, provided nothing has changed with our products.

We can then use these comparisons to verify the consistent performance of a specific product, to spot issues before we roll out a firmware upgrade, and to see how different products in our lineup perform under similar circumstances. None of this was possible with our old testing setup.

Another advantage is speed. Tests that previously took a week now take less than three hours. Better still, because the tests are automated, we can focus on other projects while they are in progress.

This streamlined workflow means we can perform more tests over the lifecycle of our products. In the past, we ran tests before we rolled out a new router or AP, and only occasionally after firmware upgrades. With IxVeriWave, we can run diagnostics on all our products at any point that they are needed.

The benefits to our engineers and our customers have been tremendous. Our spec sheets are more accurate, our gear is more reliable, and when we do encounter an issue, we can address it in less time.

Detecting and Correcting Complex RF Issues

I can give you a couple of examples of issues we were able to detect and correct using IxVeriWave. The first was a DFS—dynamic frequency selection—issue with one of our APs. The Federal Communications Commission (FCC) requires such Wi-Fi devices to detect 80% of Type 5 radar frequencies to qualify for certification, but ours fell below that threshold.

We couldn't understand why we were getting a 20–50% detection rate. After diving into the problem with IxVeriWave, we discovered that one of the radar filter parameters in our firmware needed adjustment. We were able to catch the issue before shipping the device to the FCC for certification, and we saved the time and money of going through the process twice.
 

In a second instance, we had trouble attaining gigabit throughput on a four-spatial-stream multiple-input and multiple-output (MIMO) product. We could barely reach 800 megabits per second on the 80-megahertz spectrum. In some cases, we couldn't even break 10 megabits per second. After a closer inspection with IxVeriWave, we found two issues. There was a central processing unit (CPU) load balancing issue that was limiting throughput, and a miscalibrated hardware clock causing Rx Sensitivity problems. We were able to correct the problems and boost the speed by over 60%  to 1.3 gigabits.

Shipping Reliable Products

IxVeriWave has transformed the way we test and deliver our products at Cradlepoint. Thanks to easy automation, reduced testing times, and repeatable results, we can proactively catch issues and ship products that are 100% reliable.

However, it took a while for people here to trust the measurements we were getting with our new equipment. Our engineers were used to the inconsistent results we were getting with our previous setup. But once they realized that repeatability was a feature and not a bug, they quickly came on board.

If there is one piece of advice I can offer fellow engineers who are considering IxVeriWave, it is to eliminate any external factors that may have an impact on outcomes. If your test environment isn't stable, you won't get repeatable results, even from the most accurate equipment.

Once you have created the right conditions, the sky's the limit. When your engineers start to see fast and accurate results, they will do more testing, and everyone will benefit from the increased reliability and improved quality of your products.