As a consultant who’s worked with many clients throughout the tech industry in both the public and private sector, I’ve heard the terms BDD and Test Automation used interchangeably, as I’m sure we all have. But if they’re not the same, where do the differences lay and what is the function of each in the modern testing landscape? Many people within the industry will have heard of Behaviour Driven Development, and many will believe this is test automation using “given”, “when” and “then". I've recently found myself firmly in the camp treating this layer as documentation rather than automation.
First let’s start with the question, what is better, no documentation, or unreliable outdated documentation? I’d undoubtedly go with the former every time. Unreliable documentation is the cause of bugs, inaccurate tests, and misunderstandings of requirements or intended functionality and architecture of a system. Whereas no documentation leaves no room for such misunderstanding, you must go to the single source of truth, the application itself, or the people who work on it and understand it. I’d say from experience most development teams will fall into one of these two, with the majority falling into the unreliable outdated category. This is often through no fault of their own. Most teams will start with highly accurate documentation, that gradually becomes less and less so over time, due to project pressures and deadlines meaning working code takes precedence over keeping documentation up to date.
BDD solves this issue as it is living breathing accurate documentation that must be constantly kept up to date and maintained for the code that lives underneath it to continue to pass. Whereas Test Automation is the technical implementation that allows the documentation to be tested against the application. This mindset shift is important for development teams as a whole to understand, and can save vast amounts of time by accepting BDD as the source of truth documentation, rather than keeping external documents maintained that simply repeat the functionality described in a feature file.
BDD format is written in plain English with minimal keywords exactly for the purpose of documenting functionality. This lends itself perfectly to the Agile Manifesto’s, “Working software over comprehensive documentation”, BDD Cucumber allows for your working test automation to become the systems documentation with the addition of feature files. Any organisation that writes its automation tests with a BDD layer on top of it and is integrated into a CI/CD pipeline that is run regularly.
You can almost guarantee this BDD documentation is more up to date and reliable than any system documentation you will find on hidden Confluence, Sharepoint or any documentation library the organisation may think is its source of truth for the system it’s developing or maintaining. The automation that lives underneath BDD forces this to be the case, as the instant that pipeline is red for a failing test, it's investigated. This investigation is essential to establish if the error is in the automation code, the BDD documentation or the application under test. Once fixed and that pipeline turns green, you again can be fairly certain your documentation is once again accurate.
If a client or project’s non-technical stakeholders, Business Analysts, Product Owners or Deliver Managers for example are not looking at this BDD layer as a means of story sign off, and it is not being used as the primary source of documentation, it almost certainly can be scrapped. I’m sure we’ve all seen instances of Test Engineers spending countless hours maintaining this layer simply because this is the way the project has always done things.
Needless to say, this is not a good enough reason to continue in this way.
