Real User Monitoring benefits and Limitations:

Real User Monitoring (RUM) is one of the two fundamental constituents of Digital Experience Monitoring (DEM), the other one being Synthetic monitoring. DEM is defined by Gartner as an availability and performance monitoring discipline that supports the optimization of the operational experience and behavior of a digital agent, human or machine, as it interacts with enterprise applications and services.

Limitations of RUM

Compared to more superficial approaches to Web monitoring, RUM’s attention to detail will generate for X number of users X times the data, so 100 users result in 100 times more transaction datasets. This level of precision naturally results in a more accurate diagnosis of end-user experience, both individually and in collective segmentations, but responding to specific issues can prove unwieldy if your tools lack the capacity to conduct RCA (root-cause analysis) and to generate intelligent, prioritized decision analytics — leaving the bulk of data analysis and next steps up to a likely overwhelmed and under-resourced DevOps crew. Still, no matter how basic or sophisticated your RUM solution is, it’s generally hard to complain about having too much accurate information.

A more serious limitation of RUM is this: Even when you’re continuously monitoring 100% of your real users, how can you monitor the performance of a new app function or site feature ahead of time, before re-deployment? And what about real-time monitoring during periods of low engagement, such as late at night, when there are far fewer people using your application? Is RUM the best way to check performance then?

This is where active monitoring—also known as synthetic-user monitoring (SUM) or, more commonly, synthetic transaction monitoring (STM)—reasserts its usefulness, offering a way to fill the gaps left by the more passive, real-time RUM. Synthetic monitoring works by issuing automated, simulated transactions from a robot client to your application in order to mimic what a typical user might do. These server calls and testing scripts become “monitoring” tools by running at set, regular intervals—say, every 15 minutes—and can be issued from a single designated STM client browser or from multiple browsers at different server locations to better gauge site availability and responsiveness, globally. In this way, STM can give you a steady, solid baseline on which to monitor server and application performance, 24/7, even during periods of low user engagement. Moreover, because it consists of test scripts—simulating an end-user’s clickstream through basic navigation, form submission, shopping-cart transactions, or even online gaming—STM can run in private test environments before deploying new features or during regular offline maintenance, revealing potential obstacles before real users have the chance to run into them.

But “real users” are where the limitations of STM come into the picture, because even the most thorough and prescient DevOps professional, running a regular bevy of synthetic tests, can’t predict exactly what real humans will do. This is why STM can be said to focus more on availability while RUM focuses on functionality. Any given user’s particular path through a site or app is rarely as straightforward (or problem-free) as repeated synthetic scripts like to imagine.

How often have you hit the “back” button before checking out to confirm some details about an item? Have you ever closed a check out page only to return to it later in the day—and have you ever returned directly to the page by highlighting its cached URL in the address bar as you type in the website?

These are just a few examples of common variables that interrupt a user’s intended click path—each deviation representing a possible performance issue, none of which you would be aware of (unless you’re waiting to hear about it on Twitter) without RUM.

So even though synthetic transaction monitoring can overcome some of RUM’s biggest limitations, offering a great way to fill in its gaps, there’s still far more that can be done with real-user monitoring. (Especially since RUM also provides the best way to formulate and calibrate synthetic tests.) Speaking at Velocity 2012, performance expert Patrick Lightbody suggested that a healthy balance of STM and RUM is ideal for achieving Web performance optimization goals, segmenting an ideal APM strategy into 25% internal ops and network monitoring, 25% STM, and 50% RUM. Yet to your boss and your end-users, he added, “all that matters is the external performance as perceived by real users,” which in his model would mean shifting the perceived focus on RUM to somewhere around 95%.

Benefits of Real User Monitoring:

Full visibility into software performance

Only 1% of users report performance problems. Real User Monitoring allows you to experience your apps the way your users do, so there’s no need for them to report issues for you to know what’s going on. Even simple JavaScript errors can cause a website significant problems, especially on pages with earning potential.

With RUM, if there’s a slow loading page, you’ll be the first to know about it, not your customer.

Developer productivity

In our experience, engineers spend a lot of time searching for performance problems. Engineers are expensive, and they need to spend their time creating the products that customers love, not manually crawling your web properties for issues.

Raygun, for example, will periodically crawl your app for performance problems, so your team doesn’t have to. One example of how it does this is with Insights which passes each page through a set of 22 rules, then delivers precisely which pages passed, which failed, and why.

With Insights, there’s no need for a developer to open a code editor to know there’s a problem.

Raygun passes each page through a set of 22 rules, surfacing which passed and which failed.

Track high-value pages

Prioritizing fixes is a lead engineer’s responsibility, but how do you know which problems to focus on first? Real User Monitoring allows you to monitor and discover problems on high-value pages, like a payment or signup page for example. Use the high-level charts presented on dashboards for a quick health- check of your apps.

Reduce cost to acquire customers

Most companies spend significant amounts of money on marketing efforts. But if the pages you send people to take over three seconds to load, they will just leave, wasting your marketing spend.

Real User Monitoring helps you detect these problems and fix them before they affect any more users.

Confidence in new version releases

With version usage information, see how users are adopting new features so you can make data-driven decisions about version support.

Raygun’s Version Usage page. Watch for an upward trend as more users adopt each version

Customizable data visualization

We know that performance monitoring looks different to different team members. High-level charts don’t help developers when getting to the bottom of the problem, and code snippets don’t help technical leads see trends over time. RUM tools can visualize different levels of data based on your preference, serving it in customizable dashboards for the whole team to see.

Improve customer service

It’s all about making your web and mobile experiences great for your users. RUM allows you to deal with problems much faster, most times before a user reports an issue. When a user does report a slow loading page, they will be delighted when you can get back to them saying the problem is fixed.