Ole Lensmar:
Hello everyone, and welcome to today’s episode of the Cloud Native Testing Podcast. I have the great pleasure of being joined by Daniel Bryant, who's built quite a name for himself over the years. He's currently Head of Product Marketing at Syntasso. We’ve crossed paths a few times at KubeCon and other events—always a pleasure. Daniel, I’ll hand it over to you for a quick intro before we dive in.

Daniel Bryant:
Thanks, Ole. I’d say—don’t let the title “Head of Product Marketing” fool you. My journey started more than 20 years ago in academia, working on agents—back when agents were something different than today’s AI agents. From there, I moved into Java development. I loved coding and saw more of a future in it than in academia.

Along the way, I did a lot of QA work. I think that’s actually where you and I first crossed paths. I was a huge fan of SoapUI back in the day. I lived and breathed XML for years—WSDLs, SOAP, all the craziness. Later, I moved into architecture and then platform engineering. I built a few platforms, had some successes, made mistakes, and learned a ton from some really smart folks. These days, I try to act as a bridge between the technical builder I used to be—like a CTO—and the folks who are just trying to get stuff done. I have marketing in my title now, but I really think of myself as a storyteller. I help people understand the challenges they’re facing and explore how to solve them—not just by buying our stuff, but through better approaches. Of course, if they buy something along the way, that’s great too.

Ole Lensmar:
Sounds awesome. Before we dive into testing, I have to ask—do you still run into WSDL and XML in your day-to-day work? I rarely do these days, but maybe your experience is different.

Daniel Bryant:
Oh, 100%. Just yesterday, we were having a conversation about mainframes. Some of our customers at Syntasso, especially in big banks, are still running critical systems on mainframes. They’re also using modern tools—Kubernetes, WASM—but they have a mix of everything. That means we still see a lot of Java and XML. Spring was a big part of my past, and back then it relied heavily on XML. These older systems often expose their interfaces via SOAP and WSDLs.

Myself, Chris, and Colin—two of Syntasso’s co-founders—worked together about 12 years ago as Java consultants at Open Credo in London. Whenever someone mentions WSDLs, we can’t help but laugh. It’s a shared memory. We built those systems. The younger folks on our team just give us a blank look when we talk about WSDLs. It’s a bit of an inside joke now.

Ole Lensmar:
Totally. And Maven still uses XML. In the testing world, XML is still pretty common for reporting—like JUnit report formats. I kind of miss XML schema in a weird way. There was some cool structure to it.

Daniel Bryant:
Same here. YAML feels like a poor approximation sometimes.

Ole Lensmar:
Right? But let’s not get too sidetracked. I do want to talk about your experience in platform engineering. I’ve found it really interesting, especially in the context of testing in cloud-native environments. There’s this emerging role platform engineering plays in not just enabling development, but in promoting a strong testing culture too.

Ole Lensmar:
One thing I find interesting is the role platform engineering plays in cultivating a testing culture. When you're building a platform for your development team, you might bake in some testing features. But that can feel top-down—like telling developers how to test—instead of the developers saying, “These are the tests we need,” and the platform adapting to that. What’s your perspective?

Daniel Bryant:
Platform engineering plays a huge role, and it really has to be both top-down and bottom-up. When I was a consultant, I often had to use a kind of "pincer movement"—getting buy-in from leadership while also enabling developers. At the senior level, everything boils down to a few key goals: making money, saving money, and reducing risk. Testing is a critical part of that risk reduction—things like security, scalability, reliability, disaster recovery, and business continuity.

But you can't force it on developers. They're smart—they'll work around it if it doesn’t fit their workflow. So the challenge is building platforms and tools that support their way of working and make it easy—and even enjoyable—to do the right thing. You want that harmony. Whether it’s top-down or bottom-up, it has to support speed, safety, and scalability. At Syntasso, we talk about this a lot.

Testing touches all of that—architecture, code, operations. If you’re not testing, how do you know it’s working? So yes, culture sets the direction from the top, but it has to be reinforced from the bottom up too.

Ole Lensmar:
Especially when it comes to testing culture, I’ve always been a bit cautious with blanket rules—like “You must have 80% code coverage” or “You must write these types of tests.” On the one hand, you want teams to test. On the other, overly prescriptive mandates can create resistance. How do you strike that balance?

Daniel Bryant:
Absolutely. It reminds me of Goodhart’s Law—when a metric becomes a target, it stops being a good measure. I remember back in my Java days, we were told to hit 90% code coverage. Some people gamed the system—running tests with no assertions just to hit the number. That consultant ended up getting fired, but they were just doing what they were incentivized to do.

We were using Jenkins at the time, and even though the automation was there, we needed people to validate that the tests actually made sense. These days, I prefer to work backwards from goals. What are your SLAs? What’s the user experience you’re aiming for? Start with something like the DORA metrics, and if you’re missing targets, that’s a signal you may need more or better testing.

Instead of declaring, “You must have 80% coverage,” I’d rather say, “We aim for 99.9% uptime and fewer than two incidents a month.” If those goals aren’t being met, then we investigate and add more testing where it makes sense. Metrics can be a helpful starting point—but they’re not the goal.

Ole Lensmar:
Right, and the platform should help enable that kind of thinking.

Daniel Bryant:
Exactly. All the languages and frameworks already have testing tools baked in. But the key is: how easy is it to provision and run them? A lot of our work with customers is around identifying the frameworks they want to use and then helping them build platforms that make those tools self-serve.

You shouldn’t need to open a service ticket just to get a test environment spun up. Testing should be part of the deployment process—and it should be automated. Our goal is always to make it easy to do the right thing through the platform.

Ole Lensmar:
That’s definitely something we’re seeing with our customers too. Once they start moving to cloud-native, traditional pre-artifact testing isn’t enough. They’re exploring GitOps, progressive delivery, and other patterns—and wondering how to fit testing into that.

Daniel Bryant:
Absolutely. If you can provide a platform that supports GitOps, progressive delivery, infrastructure-as-code—whatever your deployment model is—and also offers clear insertion points for tests, that’s powerful. It helps if the platform even has some opinions about which testing tools to use. That’s definitely worth discussing.

Ole Lensmar:
Exactly. Platform engineering can be a huge enabler for better testing—regardless of the framework.

Daniel Bryant:
Totally. The framework I work on is called Kratix. One thing that sets it apart is that it requires you to curate your own experience. But once you do, it’s customized to your needs.

As you said, there are key points where you know you’ll need testing: generating a software bill of materials, running resilience checks, doing security or load testing. The need is standard, but the implementation varies. We’ve found that every customer is slightly different—just enough to throw you off if you try to generalize.

Ole Lensmar:
Right—just enough of a difference to break a one-size-fits-all approach.

Daniel Bryant:
Exactly. That’s something the founders of Syntasso learned early on: every org needs a platform, but it has to be your platform. Same with testing. Everyone agrees you need it—but what that looks like varies.

That’s where we spend a lot of time—helping teams understand their requirements and baking them into the platform. Again, it’s about making it easy to do the right thing.

Ole Lensmar:
That makes sense. I imagine some people are hoping for an off-the-shelf, end-to-end system. But most of the time, it’s not going to match their architecture exactly.

Daniel Bryant:
Right. If your system was optimized for a monolith, testing is simple—you just spin it up, run your tests, and you’re done. But once you move to distributed systems—microservices, serverless—you need a different mindset. You have to test in production as well as before release. And that shift requires a platform that supports it. We spend a lot of time coaching teams through that change.

Ole Lensmar:
That brings me to something else I wanted to ask about: testing in production. It can be controversial. Some advise against it—or at least warn against using it as your only method of testing. But it’s becoming more common...

Ole Lensmar:
We’re definitely seeing more teams running tests in production as a kind of safety net. It’s similar to synthetic monitoring in some ways. There’s a fine line between the two. What we’re seeing is that teams are using testing in production to speed up their feedback loops. Instead of injecting end-to-end tests into every pull request or ephemeral environment, they might run those tests later—during progressive delivery or even post-deployment. Do you see any red flags there, or are you more pragmatic? Does it depend on the context?

Daniel Bryant:
Yeah, I think it really depends on what you're optimizing for. Startups with just a few customers can often prioritize speed. If something breaks, they fix it fast. I've seen teams that design for replaceability, do minimal pre-release testing, and rely heavily on end-to-end testing in production. That can work if the risk is low and the blast radius is small.

But if you're a big bank or airline, you can’t rely on just one approach. You need thorough pre-release testing and production checks. As Charity Majors put it, “If you’re not testing in production, your users are.” That quote really stuck with me—it’s a mindset shift. Larger orgs tend to be more risk-averse, and the phrase “testing in production” can make people nervous because they associate testing with breaking things. But it’s really about optimizing for fast feedback.

If a user can’t check their account or complete a purchase, someone needs to know immediately. It doesn’t matter if the root issue is a database or network failure—if the user experience is broken, it’s critical. So whether you call it testing, synthetic monitoring, or observability, the goal is the same: surface issues in production as fast as possible. You can do all the testing you want beforehand, but in complex systems, unexpected issues will still happen. That’s why testing at the “sharp end of the stick”—where users actually interact—is essential.

Ole Lensmar:
Right, especially in cloud-native systems, your production environment is built on an incredibly complex infrastructure. There are so many things that could go wrong under the surface—things you might not catch until they happen. Chaos testing is one approach, but even something simpler, like running end-to-end tests in production that don’t alter the system’s state, can go a long way. Running those hourly in production doesn’t mean you skip earlier tests, especially for business-critical features. But we’ve all seen bugs that didn’t show up until prod, even after everything passed in staging and progressive rollout.

Daniel Bryant:
Exactly. And today it’s relatively easy to automate those tests. Sure, there’s a cost, but it’s usually worth it to reduce risk and catch issues early.

I’ve also seen teams reuse their behavior-driven tests in production. They define tests pre-release—things like, “The system must do X”—and then run the same tests in production using the same automation framework. It’s just targeting a different environment. Have you seen that too?

Ole Lensmar:
Definitely. Some teams are designing their tests specifically to be reusable across dev and prod. It depends on how stateful the tests are. Some tests require specific data scaffolding or setup, but others are less sensitive and won’t inject large amounts of test data. Those can run safely in production without side effects.

Daniel Bryant:
Exactly. You just have to be thoughtful about test design. In ephemeral or staging environments, you can run whatever you like since it’s disposable. But in production, you need to be more cautious.

Ole Lensmar:
That brings me to something else we’re seeing more of with Testkube—specifically around infrastructure testing. In platform engineering, where teams are dynamically provisioning infrastructure, we see teams running tests on that infrastructure before deploying applications or handing it off to developers.

It could be verifying that the correct hardware is provisioned—like GPUs for AI workloads—or making sure that observability tools like Prometheus are set up correctly in a cluster. You could do this via Helm charts or other tools, but do you see this kind of infrastructure testing in your work?

Daniel Bryant:
Absolutely. In Kratix, we have a concept called “Promises” that include pipeline workflows. As you deliver infrastructure, you can add arbitrary steps—like verification or policy checks—into the pipeline. More advanced users use tools like Open Policy Agent (OPA) to enforce security rules or ensure encryption, and they run assertions to validate what’s being built.

Some also use Gatekeeper to check Kubernetes policies during runtime. We occasionally see them run checks directly on the Kubernetes cluster or namespaces that have been provisioned.

Personally, I have a love-hate relationship with infrastructure testing. A lot of infrastructure today is declarative—if you’re using Terraform or CloudFormation, your code is the blueprint. The issue is that infrastructure tests often become too tightly coupled to that code. It reminds me of working in Ruby years ago—your tests would mimic your code so closely that they became fragile. Change the infrastructure slightly, and everything breaks.

That’s why I prefer later-stage pipeline testing. You can still have policy checks earlier, but for things like verifying available CPU, memory, or storage space, I’d rather do that after the infrastructure is up. Over time, services evolve. You gradually pile more things onto a node, and one day it runs out of disk space—especially if logging is turned on.

So we advocate running post-provisioning checks to make sure what was created is actually ready for real-world workloads. It’s that balance between local validation and global verification.

Ole Lensmar:
Makes sense. Especially if you’re relying on a cloud provider that bundles in auxiliary services, it’s good to confirm that your dependencies haven’t changed unexpectedly. Maybe it goes back to what you said earlier—it depends on how critical the system is. If it’s mission-critical, you want to verify ports, versions, configurations—everything.

It’s interesting, though. Infrastructure testing wasn’t something we saw much of before. But now, since infrastructure is being treated more like application code, it makes sense to test it the same way.

Daniel Bryant:
That's it. It really requires a different mindset. Back in my early coding days, we followed the classic test pyramid. But I think that model is less valid for infrastructure, especially since so much of it is declarative now. If you’re using something like Pulumi or AWS CDK, maybe a more traditional software architecture and testing approach still applies.

But with declarative setups, you’re just describing the desired state and then asserting that it matches. When I was writing a lot of Puppet, I found my tests looked almost identical to my implementation code—and that always felt off. That’s where verifying contracts becomes important. I don’t know if you see that much these days, but back when we were building early microservices, we used a lot of consumer-driven contracts.

We even ran tests against third-party APIs to detect changes. One I remember clearly was a mailing label generator API that would randomly change. So we ran nightly tests to check if it still behaved the same. If it didn’t, Jenkins would alert us. That kind of contract assertion can be really valuable.

Ole Lensmar:
Absolutely. And I'd consider those services part of your infrastructure—or at least critical external dependencies. Whether it’s hardware or software, if your application relies on something behaving a certain way, contract-based testing, like PACT, makes a lot of sense.

Daniel Bryant:
Exactly. We sometimes live in this ideal world of design-first APIs and perfect stability, but in reality, APIs evolve. Things change. Even with our own products, we sometimes forget someone might be depending on something that seems minor. So contract verification remains important.

Ole Lensmar:
I want to circle back to something we talked about earlier—testing culture. Testing often happens late in the process. Do you think platform engineering can help teams embrace testing earlier?

Daniel Bryant:
Definitely. Platform engineering—and tooling in general—can be a huge enabler of cultural change. You need bottom-up buy-in to determine what “the right thing” is, but you also need to bake it into your tools and platforms to make it easy for people to follow through.

It’s about making it easier for the folks coming behind you—especially those who aren’t as invested in testing or security—to do the right thing. Over time, if you build the right toolset, culture will follow. You still need some top-down incentives and leadership support, of course. But in my consulting days, we saw the biggest cultural shifts when we combined good examples with the tooling to support them.

Platform engineering feels like a natural evolution of DevOps. It’s about reducing friction for everyone—especially at the infrastructure level. It bridges the gap between infrastructure and developers. When done well, it helps infrastructure teams expose their services, contracts, and testing capabilities in a self-service way that makes it much easier for developers to do the right thing.

Ole Lensmar:
Exactly. So, platform engineering can be a key enabler and driver for good testing practices in cloud-native environments. Daniel, this has been awesome. Thank you so much for joining me and sharing your insights. And thank you to everyone for listening—I hope you’ll tune in for future episodes. Take care!

Daniel Bryant:
Thanks, Ole. Always great to chat. Bye for now.