Breaking Out of the Testing Silos
The burden of “reinventing the wheel” is a common lament we’ve heard from test engineers time and again. Of course, the individual circumstances surrounding this comment can vary from company to company, but the gist remains the same. Engineers either find themselves replicating work that was successfully completed by another group, or resurrecting custom engineering that has either been dismantled or forgotten, along with the intimate knowledge of how it worked. Too often, organizationally, this cycle is set up to repeat again and again.
Overcoming the gravitational pull of the status quo is a challenge for any organization—and, for that matter, any individual. The old adage, “If it isn’t broken, why fix it?” appeals to the desire to continue risk-free, with the confidence of known output. As engineers, we revel in our ability to get the job done within given technical and business constraints. However, the pressures to accomplish the task at hand without a more holistic view of testing practices can mask systemic inefficiencies that, over time, lead to net losses.
At NI, we have the privilege of partnering with test organizations that span continents, industries, and DUTs. This bird’s-eye view of testing around the globe has surfaced the need for consistency and scalability as a consistent theme that, in some way, impacts every development cycle, regardless of company size or product scale. “Standardization” is the term we use for this need, but there is a lot of meaning packed into this one word. In this paper, we’ll explore the need for standardization, the business and technical benefits of standardizing, and how you can get begin to harness the benefits of standardization.
Standardization is a holistic approach encompassing organization structure and interlocks, technical choices, and practices, with the goal of aligning these elements to maximize the efficiency and output of a testing organization.
The Need for Standardization
In our modern world, collaboration has never been easier. And while technology has helped to break down location and time-zone barriers, organizational structures have not always kept pace. Often, teams are siloed. With on-time product delivery as a driving metric, teams are tasked with managing project budget, design, and testing as they see fit to achieve product and timeline goals. This leads to individual successes, but fails to capitalize on the greater gains of shared best practices, transferrable talent, and overall testing robustness that a top-down test strategy offers. In response to this misalignment, standardization is born. Standardization is a holistic approach encompassing organization structure and interlocks, technical choices, and practices, with the goal of aligning these elements to maximize the efficiency and output of a testing organization.
Taking a step back and assessing your organization’s engineering inefficiencies can help illuminate how standardization can be particularly impactful to you. Here are some questions to consider:
- Is there effective technical collaboration and best-practice sharing between teams?
- Can individuals easily move between groups as needs arise, with minimal ramp-up?
- Is data directly comparable from one site to another?
- As your organization grows, are you poised to quickly scale test equipment across geographies?
- As talent moves on from your organization, how devastating is the loss?
Company culture and historical operations set the stage for how you answer these questions. Regardless of your organization’s history and the perceived cost of change, an honest assessment of these areas reveals critical pockets of opportunity. Now, we need to understand not just why standardization is needed, but what it brings to the table.
Demystifying Standardization: What Does It Really Do for You?
Though NI gets to work firsthand with leading test organizations around the globe, it’s surprising how often we connect groups within a company that have no idea of their overlapping goals. This problem is multifaceted; encompassing technical choices, company culture, and company organization. Without much context, most people would agree that the idea of standardization is likely useful, but what does it really do for you? Let’s look at a few scenarios.
Technical Choices: Standardization Enables Reuse
With teams creating their own test systems and defining unique approaches to test, sometimes innovative approaches are not sharable between groups, even if the goals are the same. Standardizing on consistent technology allows teams to collaborate across product development steps, regardless of team, site, or development stage. Groups can seamlessly share best practices and helpful customizations that don’t need to be recreated at each step.
Company Culture: Standardization Amplifies Your Engineering Talent
An agreed-upon, ubiquitous approach to testing mobilizes engineers to float between projects, teams, and sites because the technology and best practices are consistent. While this is not only beneficial for quickly balancing resources, it also transforms an organization by positioning engineers to be a unifying force of company culture and is critical for scaling as your organization grows. Teams still can have localized goals, but inherently cease to be siloed from one another with this approach, as engineers are positioned to collaborate, troubleshoot, and share knowledge across organizational boundaries. Additionally, engineer mobility simplifies the hiring process, as there is an agreed-upon skill set across the organization.
Company Organization: Standardization Simplifies Dependencies
Standardization forces holistic, top-down thinking. Buy-in at the appropriate leadership level cascades down to vendor choices. A streamlined approach to testing eases the external dependencies that an organization manages. By leveraging a select list of vendors that work well together, your company will manage a limited number of relationships. This simplifies support needs and troubleshooting and, if the vendor list is well-curated, ensures a globally consistent support structure.
All of these facets—technical choices, company culture, and company organization, seen through the lens of standardization—lead to efficiency gains, a reduced learning curve, cost savings, reduced development time, and reduced time to market, and give a sustainable maintenance path.
How to Standardize
As discussed, standardization starts with a top-down approach. The bird’s-eye view is necessary to revolutionize the status quo. No matter your level, from VP to engineer, you can be a catalyst for change by elevating the benefits of standardization and proactively looking for connection points within your organization. And NI is here to help. We often are the glue between disparate groups, and are happy to amplify conversations, as well as share anecdotes from a global perspective. Our Methodology Consulting Services (MCS) can help assess your needs and develop your organization’s most rewarding path forward. Examples of MCS include defining the correct ROI and metrics for different parts of your organization, as well as setting clear governance, policies, and standards that safeguard standardization efforts and ensure optimal return.
In addition to organizational change, the technical standardizing choices you make are critical. As the word implies, you are choosing one way of doing things at the exclusion of others, so there is pressure to choose well. One way you can ensure you’re not held hostage by your standardization choices is to go with an open system that provides structure, but still allows you to customize as needed, leveraging proven reference designs. The NI system hinges on this point. With modular hardware, capable of virtually any measurement, that is also software-defined, NI can cover the breadth of testing you need while still offering customization of almost every product. You can maintain your expertise without relying on any one vendor to debug or customize.
An open, integrated system also lets your organization define your unique best practices and leverage completed work. Because you can incorporate code from your language of choice (such as Python) and integrate third-party instruments, NI doesn’t corner you or force you to sacrifice years of technological investment as you heighten collaboration and output.
This openness consequently fulfills testing needs from development to production: You can swap out modular hardware under new design requirements or test types and make rapid changes throughout the development process because of a consistent software API. You are free to choose how much out-of-the-box, configuration-based functionality works best for you, and how much customized, to-the-pin definition you’d like to implement.
Standardization touches nearly every facet of a test organization’s operation, from people to technology. It’s tempting to continue with the status quo, rather than engaging in such a transformational endeavor. However, standardization becomes the bedrock for future growth and scale, while presently saving you development time and budget. Relying on a trusted partner such as NI ensures your success. And utilizing an open environment that amplifies your expertise and welcomes your preferences ensures that standardization is never a limitation, but rather, a means to unlimited potential.
See How the NI Platform Spans Both Design and V&V Test