Quality Phases
For each of our critical component updates, Trend utilizes a multi-layered safety net process consisting of four (4) distinct quality phases – three (3) of which are mandatory and are included in every release, and the last one (recovery) only being invoked in the rare circumstance where something does make it into post-deployment.
In each of these phases, Trend has a variety of both automated and human-based testing, checks and verifications to ensure that all updates meet stringent quality, compatibility and performance checks – with one of the most important being stability. Most of these process have been built using industry-leading standards and several are certified by 3rd parties (e.g. ISO).
While not exhaustive, below are some key processes Trend utilizes for different types of updates. If an error or red flag is detected during any of these stages or processes, the release process can be halted to prevent a flawed update from being deployed.
Content Updates (e.g. Patterns & Rules)
Various security content updates - also known as patterns, filters and rules - are some of the most frequently updated components in many of Trend’s products and services. These updates typically deal with proactive protection against different kinds of emerging threats. The nature and frequency of these updates make pre-release testing critical.
Development Phase:
- Initial Quality Assurance (iQA) – includes pattern code review for instruction and logic errors, as well as actual threat coverage and matches against known OS and verified files.
Testing Phase:
Product and Feature Updates
Due to the nature of security software, it is necessary to operate in some capacity at the kernel level to provide adequate protection against new threats. Security developers must weigh benefit and risks with how they utilize the kernel space. For example, while utilizing the kernel space for components such as communication and updates may appear to make an agent appear more efficient, failures - even minor ones - could prove catastrophic.
Each of Trend’s comprehensive quality processes are interconnected to allow potential critical errors to be identified and resolved before they can impact any of our customers. While no set of processes can prevent 100% of all issues, our commitment to continuous improvement of our internal prevention policies and ability to mobilize worldwide resources to recover and assist customers, when necessary, can provide the utmost confidence to our customers.
- Various build testing – includes several tests around product integration for different Trend products, performance testing, actual detection/protection, and regression testing.
- Internal deployment – also known as “eating our own dog food” – Trend deploys updates in internal environments during the testing phase, before pushing externally, to ensure releases meet production quality standards.
- Phased deployments – updates are released in different phases for more efficient delivery globally and can be halted if any issues are reported during any of the phases.
- Takedown/rollback - in the unlikely event that a pattern issue is discovered post-deployment, Trend has mechanisms in place that allows us to halt and remove a pattern and revert to a last-known good configuration that can be deployed in a much quicker timeframe. This type of scenario in most cases will occur for a reported issue that may not necessarily be widespread across multiple customers but still may have a business impact.
Product and Feature Updates
While not as frequent as content updates, product and feature updates must undergo several quality process checks. Since these updates are generally more complex, the quality tests reflect this additional complexity.
Development Phase:
Development Phase:
- BSOD SafetyNet – Trend has implemented testing methodology and safety mechanisms that specifically attempt to identify driver and component issues that may cause the dreaded “Blue Screen of Death” (BSOD) or similar type of critical error issues and attempt to disable the driver during the boot process in the testing.
- In-house Performance and Compatibility Tests – utilizing comprehensive testing labs and setups, new updates are tested against predefined configurations and scenarios that match the most common and critical customer deployments.
- Dogfooding – as with the content updates, Trend deploys product and feature updates internally to ensure production readiness before rolling out globally.
- Ring Deployment (Trend side) – Trend utilizes a ring methodology of phased deployment for these product and feature updates. These rings include internal Trend deployments (“dogfooding”), early adopters, general availability, and a ring for sensitive / mission critical environment deployments. These are managed on the backend by Trend Micro.
- Customer Configurable Options – Trend provides customer configurable options such as content control versioning for both product and patterns, and scheduled tasks. These options allow customers to design their own rollout plans for environmental stability.
- In the event a post-deployment issue is reported, Trend has processes to quickly remove potentially impacted updates and replace with either last-known good or fixed updates as quickly as possible.
Trend’s Kernel Philosophy
Due to the nature of security software, it is necessary to operate in some capacity at the kernel level to provide adequate protection against new threats. Security developers must weigh benefit and risks with how they utilize the kernel space. For example, while utilizing the kernel space for components such as communication and updates may appear to make an agent appear more efficient, failures - even minor ones - could prove catastrophic.
Because of this, Trend's design philosophy when it comes to kernel mode is to favor safety and stability above all else. We understand that operating at this level on a system carries great responsibility, one which we take very seriously. In practice, this means that we only put necessary components in kernel space and will try and utilize user mode whenever possible to minimize risk.
Trend’s Ongoing Commitment to Quality
Each of Trend’s comprehensive quality processes are interconnected to allow potential critical errors to be identified and resolved before they can impact any of our customers. While no set of processes can prevent 100% of all issues, our commitment to continuous improvement of our internal prevention policies and ability to mobilize worldwide resources to recover and assist customers, when necessary, can provide the utmost confidence to our customers.