Synthetic Transaction Monitoring: Guide & Best Practices

Synthetic Transaction Monitoring

Identifying problems with your website or software before end-users notice performance issues is crucial for long-term success. Discovering issues as they arise, however, isn’t easy even with dedicated QA resources. This is especially true when your products have complex functionality that relies on high-performance APIs and third-party dependencies. In this article, we’ll explore how you can address the issue of testing-at-scale using synthetic transactions.

What are synthetic transactions?

Synthetic transactions are the result of scripts simulating activity that would normally be performed by real users on an application or website. These transactions are primarily used to test production environments’ performance, but they are also useful for testing the functionality of updated applications prior to deployment.

For example, you can create a script that simulates a user’s connection to your website via a defined entry point. This script could also include any subsequent actions or navigations as a follow-up. In this case, you could use the script to validate that all users have explicit access to specific resources, regardless of their location or entry point to your services. You could also use it to validate user roles and permissions.

Another example is simulating database connections. Using synthetic transactions that run at set intervals lets you see how your website or database would perform under various loads that reflect real user activity.

Synthetic transactions vs. API monitoring

The main similarity between synthetic transactions and API monitoring is that both rely on software rather than real people. Also, they both verify if the application performs as expected and delivers a quality user experience.

Here’s how they differ:

• Synthetic monitoring: Mimics application behavior through simulations and emulations.
• API monitoring: Validates the internal and third-party APIs used for communication.

Synthetic transactions monitor applications that may use APIs. API transactions monitor an API usable by many applications. Together, they provide a comprehensive view of performance across a transaction’s full workflow.

Best practices for synthetic transactions

If you are just starting with synthetic transactions, here are a few important tips and best practices for you to follow when building out your synthetic transaction library.

Define a naming convention for transactions

Hunting for the right monitor, especially when you are in a hurry, can get pretty overwhelming. It’s a common problem, especially if you have multiple monitors that target the same service. For example, let’s say you have a ping check that tests availability every minute. Then, you have a monitor that tests the full user workflow. These two are obviously different tools, but you can hardly tell them apart if they target the same production services.

To avoid this issue, adopt a transaction naming convention based on clear, precise guidelines. For example, a convention that’s based on Type – Service – Env would translate into Ping – Webstore – Production as your synthetic transaction’s name.

Assign permissions to protect sensitive data

Setting up role-based access (i.e., permissions) to sensitive data is optional, but encouraged. Granting all internal users access to all transactions is not ideal, especially when some scripts can contain or fetch sensitive data. For example, some synthetic monitoring tools lack the ability to obfuscate credentials, which may affect how you assign role-based access.

The more monitors and users your organization has, the more important it is to use permissions. Role-based access is typically based on user groups. Restrict groups using the least privilege principle to segment access to your synthetic transaction library. That will help you better protect sensitive data and provide a secure experience.

Monitor third-party applications

Many applications rely on external APIs to enhance their user experience. Weather information, music, and calendars — all of these require your services to communicate accurately with third-party applications. Submitting synthetic transactions to these external endpoints can help you monitor third-party applications and detect important changes, such as major or minor API version updates that affect request and response structures.

Synthetic transactions also enable you to monitor your vendors’ performance and detect SLA breaches. Having unbiased historical data on record is essential to confidently holding your vendors accountable to their commitments.

Test new features

Of course, synthetic monitoring is important at any stage of development. However, it’s exceptionally useful for testing new features before your website or application goes live. Synthetic monitoring is an ideal way to discover potential errors and obstacles users might encounter. For example, synthetic transactions can quickly validate that all input fields for a given feature accept the correct input type and list of all expected enumerations.

Test from edge to edge

As edge computing expands its reach, it becomes increasingly difficult to test all relevant locations without synthetic monitoring. For example, you may need to check potentially thousands of edge points of presence to validate that a critical update was received. That’s what makes automated monitoring edge-to-edge so critical.

Testing from multiple locations ensures end users have both access and acceptable interaction times with your applications. For example, say you’ve added a new store location to your app, which users can choose when browsing available goods or selecting pickup options. You can use synthetic transactions to test that the newly added store population in the “stores near me” results in Google and that all shopping options are available. At the same time, remember to test regions that represent not just the core of your user base, but also any emerging regions where usage is increasing.

Setting up synthetic transaction monitoring with a scope similar to your reach is the only way to fully validate continual first- or third-party updates.

Set alerts

Define performance thresholds

Define thresholds for the performance and availability metrics of your synthetic transaction so your team can quickly identify problematic trends. Define both globalized (all locations violating) and localized thresholds.

Choose alerting pathways

You can get notified of the performance drop in numerous ways. For example, your alerting pathways can utilize SMS, email, push notifications on your mobile device, or even an application such as Slack, OpsGenie, or PagerDuty.

Set up role-based ownership/urgency labels

Segmenting alerts by specific groups can also be helpful if application ownership is divided among different engineering or product teams. For example, a CDN alert can be routed to a dedicated CDN team, while a problem with a social tag can be routed to a campaign manager.

Setting up role-based ownership and urgency labels helps employees understand what they need to pay attention to. Channels that are too noisy may get less attention. Dedicated alerts, however, feel more urgent.

Avoid test dependency

Whenever possible, ensure your synthetic transactions can run independently and in any order. In other words, avoid test dependencies in order to unblock your vision of how the whole application is performing when a given test fails. It also allows you to run multiple tests simultaneously without waiting for other tests to complete.

Pitfalls for synthetic transaction testing

This section is dedicated to some of the challenges and pitfalls you might face when managing synthetic transaction testing.

Captchas

Many websites use captchas, which are designed to prevent the type of automated (bot) monitoring that is at the core of synthetic transaction monitoring. This makes it harder to rely on synthetic transactions for performance data. You can get around captchas by either disabling them in your test environment or adding a hook to allow tests to bypass them.

Two-factor authentication

Two-factor authentication (2FA) works by generating a one-time password via mobile apps, SMS, or email. It is difficult to factor into automated testing.

You can disable 2FA in a test environment, for certain users, or for users who log in from specific locations (via IP addresses). However, given the importance of data security, it’s highly recommended that you select a synthetic monitoring tool that can incorporate 2FA as part of your testing criteria.

Social logins

We don’t recommend logging into popular websites such as Facebook and Gmail using tools like Selenium’s WebDriver API. Not only is it slow, but you can also get your account banned.

It’s much better to use the APIs that email providers offer, or the Developer Tools Service for Facebook logins. It might seem like extra work to become familiar with the APIs rather than logging in via the UI, but in return, you will get greater speed and reliability.

Key functionalities of synthetic transaction (monitoring) tools

Now that we’ve gone over the do’s and don’ts of synthetic transactions, let’s review what features a good synthetic transaction monitoring tool should have.

• Support for multiple types of logins: Enables you to log in using SSO, Citrix, XenDesktop, AWS Workspaces, etc.
• Simulation of different protocols: Enables you to leverage REST, JSON, YAML, HTTPS (web), SMTP (email), ICA (Citrix), SQL (database) for various situations.
• Record and replay: Enables you to record a user-driven transaction and replay it instead of writing a script. Revision to certain classes, without re-recording the whole transaction, should also be possible.
• Smart element locator: Helps you find UI elements as the design evolves over time.
• Conditions: Enables you to direct transactions to wait for the completion of another step or even loop the transaction a certain number of times.
• Support for various types of variables: Enables you to dynamically reuse tests based on data, such as location.
• Reusable steps: Enables you to reuse and reassemble simulation components to build new tests.
• Continuous integration: Enables you to run scripts triggered by events in your DevOps continuous delivery processes.
• Assertion and Validation: Enables you to validate a Boolean expression to be true or false before proceeding. The assertion validation can be a 200 HTTP response, a JSON payload, or an image or text on the website. You also need a Safe Harbor feature to make sure that content complies with privacy laws in a specific country. In addition, assertions can be used in the absence of certain elements, such as when an item is out of stock on e-commerce websites and in stores.
• Step-by-step measurement and completion events: Enables you to record a separate measurement for each step. The completion event, for example, lets the synthetic monitoring node know when the monitoring is complete.
• Step-by-step analytics: Enables you to evaluate the performance and availability data for both an entire transaction and its individual steps.
• Payload execution time extraction: Enables you to execute custom payloads and extract their execution times separately from your total transaction times.
• Support for Selenium: Enables you to quickly compose tests referencing a list of functions and examples that populate the IDE.
• Data grouping: Enables you to compartmentalize data subsets into themed measurement units (e.g., tied to a CDN or to infrastructure)

Conclusion: Using synthetic transactions is critical

As you can see, using synthetic transactions for your website or application is crucial for optimal user experience. Synthetic transactions ensure not only the health of your application in a production environment, but also the validity of the external endpoints and data it requests across your features.

Moreover, synthetic transactions enable development and pre-production QA testing through an extensive library of transactions that test both basic functionality and performance under high loads. In other words, when you use synthetic transactions to test before you implement releases, your releases will be of increasingly higher quality over time.

FAQs