This article was originally published by TestProject. Thank you for supporting the partners who make SitePoint possible.
As we enter the last quarter of 2017, TestProject’s team decided to round up the best open-source test automation frameworks out there, to help you choose the right one for you!
Here are the pros and cons of 7 different open-source test automation frameworks.
1. Robot Framework
Robot Framework (RF) is a test automation framework for acceptance testing and acceptance test-driven development (ATDD). This framework is written in Python, but can also run on Jython (Java) and IronPython (.NET), and therefore is cross-platform (Windows, Linux, or MacOS).
It simplifies the test automation process by utilizing the keyword-driven testing (KDT) approach, which helps testers create readable tests that are easily made.
Has easy to use test data syntax.
Has a rich ecosystem around it consisting of various generic test libraries and tools that are developed as separate projects.
Has many APIs that make it highly extensible.
Although it’s not a built-in ability, RF enables to execute parallel tests via pabot library or Selenium Grid.
It’s not easy to customize HTML reports.
Bottom line: This cross-platform framework is highly recommended if you are targeting for KDT automation with a wide range of libraries and extensions. If you want to add new keywords (via RF test library APIs), a basic knowledge in Java/Python/C programming language is required.
JUnit is a framework for unit testing of Java applications, used to write and run repeatable tests.
Tests are written in pure Java which is known as the leading programming language worldwide.
Supports test-driven development (TDD).
Enables you to create your own unit test case suite.
Integrates very well with other tools (for example, Maven) and with IDEs (for example, IntelliJ).
Has history – so it has a large user base that makes it easy to find documentation on it.
If a mocking ability is required, one needs to add Mockito (or some other mocking library).
Tests are not readable by non-technical people, since for example the methods names in JUnit are constrained by Java conventions.
Bottom line: If you’re looking to write unit testing for your Java application, it’s probably the best choice. However, for functional testing or non-Java applications, you should consider other solutions.
Spock is a testing and specification framework for Java and Groovy applications. It is based on JUnit.
Creates readable tests and supports plain English sentences, making it easy to read.
Provides the surrounding context, so it easily enables you to understand how to fix a failure.
Has built-in mocking and stubbing capabilities.
Supports data-driven-tests (DDT).
Requires a basic knowledge of the Groovy programming language.
Bottom line: If your application is based on JVM and you are targeting for BDD test automation with DSL, this framework is just for you!
NUnit is a unit testing framework for all .Net languages. Originally inspired by Junit, it is written entirely in C#, and has been completely redesigned to take advantage of many .NET language features.
Quick initiation and test execution.
Comes with assertions and annotations.
Enables parallel testing.
Supports test-driven development (TDD).
It is not cross-platform as it is used only for .Net languages.
It doesn’t integrate into the Visual Studio ecosystem, so using it means more maintenance.
Bottom line: A nice open-source framework for C# unit testing, with a long history and good reputation. However, in case you’re already using .NET languages, you may consider MSTest.
TestNG is a test automation framework for Java that is inspired by JUnit and NUnit, but includes improved and new functionalities (NG – Next Generation). It is designed to cover all the test automation categories: unit testing, functional test, end-to-end, integration testing, etc.
Continue reading %7 Open-Source Test Automation Frameworks%