Agile projects are known for having shorter milestone cycles (1 to 4 weeks) and smaller releases. Quite often, or even always, the scope of work can change. These short cycles mean that a lot of performance testing is accomplished at the end of the sprint when sprint testing is otherwise planned. This blog is going to explain how to perform performance tests on agile projects using two different approaches:
Whether you are a developer or a tester, there is always room to improve performance. When I say performance I’m not referring to the speed at which a page loads but the speed at which your team is able to complete work (you can learn more about the definition of performance in my Agile Performance Explained series). While evaluating performance as a team is hard, comparing against other teams is easy. That’s why I’ve created this list of Agile Performance Testing Tools that you can use to compare against others.
Worksoft
Worksoft, the industry leader in continuous test automation for enterprise applications, continues to be the top choice for global enterprises looking to accelerate Agile-plus-DevOps initiatives and achieve true end-to-end business process automation at a massive scale.
Worksoft Certify provides an Agile testing framework that enables non-technical users, developers and test automation professionals to work together in an Agile fashion and easily integrate into DevOps tools, chains, and processes.
Key Features:
- Enable Agile adoption by building automation closer to development sprint when documentation is top of mind
- Quickly identify and document existing business processes and variations
- Achieve end-to-end business process testing across enterprise applications
- Efficiently keep pace with complex application landscapes and frequent application updates
- Uncover and resolve data transfer problems and system issues before production outages occur
#2) PractiTest
PractiTest is used as a test management tool for agile testing. It is easy to learn, affordable & flexible and helps both the developers and testers. PractiTest includes requirements, test runs, results, issues, reporting and provides the detailed status of your project. It manages development and testing processes and has control of each and every task.
PractiTest provides information to different stakeholders – developers, testers, senior management, etc in the way in which they require. It has the capability to display information in multiple ways at a glance.
Key Features:
- It can be integrated with tools like JIRA, Jenkins, Selenium, TestComplete etc.
- It has a hierarchical tree structure to manage and find information.
- The powerful and customized dashboard provides relevant and accurate information.
- Easily imports existing data.
- Complex database queries can be easily generated.
- Creates manual tests and organizes based on sprints.
- PractiTest has the capability to create bugs directly from the test run.
#3) JunoOne
JunoOne is a Great tool for Agile Test Case Management and Issue Tracking with powerful JIRA integration.
Work efficiently with sophisticated Test Case Management system JunoOne, designed to streamline Test Management and Incident management. Test Management Software JunoOne offers a number of tools that will make all the testing activities well arranged, help organize your work, control the individual steps, and the overall state of the projects.
JunoOne offers several features throughout the whole process. It starts with creating the campaign and performing test analysis. Then you plan to test and run the tests. Last but not least, it solves the issues and protects your data.=> Visit JunoOne official website
#4) JIRA
JIRA is an Agile Testing as well as a Project Management Tool developed by Atlassian. Atlassian Corporation Plc is a software company that develops products for project managers, software developers, etc. Atlassian was launched in 2002 with its headquarter in Sydney, Australia.
Its revenue was around $620 million in 2017 with a total employee count of around 2,200 and serves around 60K customer across the globe. Atlassian is well known for its tool – JIRA which is used for defect tracking, planning, reports, and managing all agile software development projects.
Key Features:
- JIRA supports an agile methodology like Scrum, Kanban, etc.
- It has a strong reporting feature which provides access to dozens of reports with real-time team performance.
- Plans and forecasts roadmap and are able to make informed decisions.
- Customized filters using JIRA Query Language.
- It can be integrated with the developer tools for end to end traceability.
Visit the official site: JIRA
#5) TestRail
TestRail is a modern Agile Testing Tool owned by Gurock Software Company. Gurock was launched in 2004 with its headquarters in Berlin, Germany. Gurock Software specializes in software development and QA. Products developed at Gurock are helpful for customers worldwide to build their software.
Currently, many large-scale organizations such as Microsoft, Dell, Oracle, Intel, HP, Adobe etc are using their products. TestRail is a web-based tool used by the testing team to manage Agile Testing Efforts. TestRail is specifically designed for the purpose of supporting an agile project milestone using sprints, various reports, and metrics.
Key Features:
- TestRail manages test cases, test runs, track test results and manages entire Testing efforts.
- TestRail integrates with automation tools like JIRA, Jenkins, and Selenium.
- Using its cloud edition, you can use it without installation or set-up.
- Easily selects test cases based on the filter for execution and stores the result using the modern interface.
- Various dashboard, reports, and email notifications keep the user informed throughout the testing cycle.
- It can be integrated with Bugzilla, GitHub, Rally, TFS, FogBugz etc.
Visit the official website: TestRail
#6) SoapUI
SoapUI is an agile testing tool and is the most advanced REST and Service Oriented Architecture (SOA) which was developed by SmartBear. SmartBear is a Software Information Technology Company which was launched in 2009 with its headquarters in Boston, USA.
SoapUI is basically used for functional testing of web services. This includes web service development, invoking of web services, etc. Using SoapUI, Test cases are easily created for web services. It has strong a GUI which helps to execute tests and analyze the reports in the GUI itself.
Key Features:
- SoapUI is a free and open source tool.
- SoapUI supports SOAP web service functional testing and REST API functional testing.
- It allows to create and execute functional, regression and load tests.
- Test cases can be easily created using drag and drop interface.
- SoapUI provides complete test coverage and supports all standard protocols.
- It supports multiple environment and data-driven testing.
Visit the official site: SoapUI
#7) VersionOne
VersionOne was launched in 2002 but as of 2017, it is acquired by Collabnet Inc. VersionOne develops agile software solutions and services for fortune 100 companies. It is headquartered in Alpharetta, Georgia, and other offices are in Atlanta and Amsterdam.
VersionOne is an agile testing tool as well as an Agile project management tool to support agile methodologies such as Scrum, Kanban XP, Lean, etc. It provides a centralized view of the backlog with drag and drop interface.
Key Features:
- VersionOne provides centralized project management and improves the project visibility.
- Stories and epics can be managed across multiple projects and project teams.
- Easily defines and tracks delivery across all the projects and project has the visibility across the entire project lifecycle.
- It allows the user to prioritize stories and defects using drag and drop interface.
- VersionOne can be integrated with JIRA, Jenkins, TFS, GitHub, etc
Visit the official site: VersionOne
#8) Selenium WebDriver
Selenium WebDriver is an agile automation tool which is widely used across the software industry. In today’s technology, the word “Agile” and “Automation” are used frequently due to their huge advantages.
It is truly required in the software industry to run any project in agile methodology and run repeatable tests using automation – Selenium WebDriver for a web-based application.
Selenium WebDriver is used very often in many agile projects as every team member can contribute to automating the test cases.
Key Features:
- Selenium WebDriver is used only to automate browser-based application and it does not support desktop based applications.
- Keyword driven framework can be easily built using Selenium WebDriver
- Simulates how an actual user can interact with the application using automated test cases.
- It supports JAVA, C#, Python, Ruby, PHP programming languages to write automation scripts.
Visit the official site: Selenium WebDriver
#9) JMeter
Apache JMeter is an agile performance testing tool written in JAVA language. It is designed to measure the performance of an application and its functional behavior under a specific load. Using JMeter you can simulate a heavy load on the web server and analyze its performance.
Key Features:
- JMeter is an open source tool.
- Graphical analysis is possible for the performance measurement of the application under different types of load.
- Ability to perform Load and Performance testing on different applications, servers etc.
- Extended plug-in support for data analysis and visualization.
- JMeter can be used for static and dynamic resources like Servlets, Java Objects, and FTP servers to measure their performance.
Visit the official site: JMeter
#10) qTest Scenario
qTest Scenario is a JIRA plug-in which is used for agile testing. It helps the project team to take a test-first approach in agile testing. qTest Scenario manages the creation of test cases, execution, defect tracking and results.
Key Features:
- It is built specifically for BDD, TDD and ATDD approach.
- Can easily import and export the existing files.
- It provides Gherkin editor for scenario and feature development.
- Test scenarios can be executed using manual method through JIRA interface or using automation through Maven or Ruby.
- Analyzes test coverage and features.
- It can create custom reporting using JIRA and can compare velocity reporting.
- For automated test results, it can be integrated with Cucumber via Maven.
Visit the official site: qTest
#11) Pivotal Tracker
Pivotal Tracker is a project management tool that is used for the purpose of project planning. It is based on Agile development methodology, however, it works well in all types of project models such as the Waterfall model, Iterative model, V-model, etc.
Tracker also helps in iteration planning during agile testing. It helps to break down a large project into the small tasks or chunks and can be managed and delivered easily. The agile team delivers the project consistent with the help of a tracker.
It helps to get the collaborative view of the entire team and analyzes the progress of the team at the end progress of the project. It shows project status at a glance.
Key Features:
- Shared view of the team helps to prioritize the project task.
- Easily visualize scope and focused teamwork.
- Project team gets the real-time progress view and shows team status.
- Can get the clear and single view of who is responsible for which task and what is a team member’s next task.
- It helps to keep a track of all the stories from the project start to its successful delivery.
- Pivotal Tracker provides configuration updates.
- Powerful Search support in the tracker also saves the regular searches to use it later when required.
- Easy drag and drop facility to attach the files, presentations, etc
Visit the official site: Pivotal Tracker
#12) Zephyr
Zephyr is a real-time Test Management Tool which was developed considering today’s technical era, newly developed complex business applications, and was designed to satisfy dynamic changes in the testing processes. It manages all processes, task associated with the software testing lifecycle and delivers a high-quality product.
Along with the agile methodology, Zephyr is useful for various other methodologies such as Waterfall, V-Model, etc. Zephyr is integrated with JIRA so that testing activities are managed easily. With this Zephyr add-in into JIRA, you can track the project status and deliver a quality product.
Key Features:
- It can be integrated with JIRA, Selenium, Jenkins, QTP, Bamboo etc.
- Zephyr for JIRA is integrated within JIRA such as look and feel similar to JIRA.
- Imports user stories and maintains traceability inside Zephyr for JIRA.
- Tests can be created & modified and also plans test execution cycles.
- Tests are organized in structure folder management so that they are easily reusable.
- Zephyr builds test cycles, aligns sprint and assigns resources for execution.
- Executes tests, stores test results into Zephyr and easily tracks quality metrics.
- Zephyr Enterprise provides all information such as project, sprint, and release at a glance
Visit the official site: Zephyr
#13) QMetry
QMetry is an Open source agile testing tool and was also developed for DevOps. QMetry helps the agile team to build, manage and deploy the software faster as compared to other tools. It provides an agile solution with powerful test management, customized metrics, test automation and Continuous Integration.
QMetry provides a plug-in for JIRA and using this tool it provides advanced test metrics, reports to the JIRA users. QMetry helps automation for Continuous Integration and Continuous Development. It converts automation results into quality metrics and detailed summary report.
Key Features:
- QMetry is integrated with ALM, Rally, Bamboo, VersionOne, Selenium etc.
- It supports agile testing to create and organize a large volume of user stories and test cases.
- Enables test planning, execution, defect management, traceability and complete coverage using agile testing.
- QMetry for JIRA is designed for test management activities for agile projects.
- Drag and drop interface can link user stories to the scenarios.
- Customized dashboard, detailed analysis reports provide progress on the testing activities.
Visit the official site: QMetry
Agile Test Automation Tools
Effective Agile test automation tools support −
- Early test automation using a test-first approach.
- Writing test automation code using real languages, domain specific languages.
- Focusing on the expected behavior of the system.
- Separating the essence of the Test from the implementation details, thus making it Technology independent.
- Fostering Collaboration.
Automated Unit Tests (using Junit or NUnit) support test-first approach for coding. These are white-box tests and ensure that the design is sound, and that there are no defects. Such tests are built by developers with support from testers, and can be independent of the functionality that is required. This results in delivering a product that may not meet customer requirements and hence with no business value.
This concern is addressed by automating Acceptance Tests that are written with collaboration of customer, other stakeholders, testers and developers. The automated Acceptance Tests are written by the customers or product owners/business analysts reflecting the expected behavior of the product. The developers’ involvement ensures the production of code as per the requirements. However, if the testing is focused only on acceptance, the resulting code may remain non-extensible.
Thus, Automated Unit Tests and Automated Acceptance Tests are complimentary and both are needed in Agile Development.
Agile Tools and Frameworks that support Automated Acceptance Testing are −
- Fit
- Fitnesse
- Concordion
- Ruby
- Cucumber
Fit
Ward Cunningham developed the tool Fit that can be used for Acceptance Test Automation. Fit allows −
- Customers or Product Owners to give examples of product behavior using Microsoft Word and Microsoft Excel
- Programmers to easily turn those examples into automated tests.
Fit 1.1 supports both Java and .NET.
FitNesse
FitNesse is a wiki, which is a style of web server that allows any visitor to make any edits, including changing existing pages and creating new pages. A simple markup language lets you easily create headings, make text bold, underline, and italic, create bulleted lists, and do other kinds of simple formatting.
In FitNesse, Acceptance Test Automation is as follows −
- Express tests as tables of input data and expected output data.
- Use FitNesse to put the test table on the page that you can edit.
- Alternatively, put the test table in Microsoft Excel, copy to clipboard and then use the Spreadsheet to FitNesse command to have FitNesse format your table properly
- Run the test
- You get the test results by color coding of the cells in the test table
- green cells represent that the expected values are obtained
- red cells represent that a different value than what you expected is obtained
- yellow cells represent that an exception was thrown
Cucumber
Cucumber is a tool based on Behavior Driven Development (BDD) framework. The key features are −
- Is used to write acceptance tests for web applications.
- Allows automation of functional validation in easily readable and understandable format like plain English.
- Was implemented in Ruby and then extended to Java framework. Both support Junit.
- Supports other languages like Perl, PHP, Python, .Net etc.
- Can be used along with Selenium, Watir, Capybara, etc.
CONCLUSION
Performance testing is hard to do right. There are many different tools and methodologies out there, so it can be daunting to choose what is right. In this post, I want to share the tools I use and why I think they are useful.
If you work in Agile, you’ve probably been assigned to test sometime during your career. On my teams, developers are responsible for their code but QA is responsible for testing the product. This means at least three things: Identifying and reproducing bugs and issues, communicating with the development team to report bugs, meeting numerous times a week if not every day (sometimes called stand-ups or scrums) for timely reporting of assigned tickets, and also defining the requirements for each ticket.