Boost Open Source Code Quality with EarlyAI
At a Glance
The Challenge:
- Time-consuming test creation processes, whether manually or automated
- Mediocre quality results from creating tests with ChatGPT
- 10-15 minutes to create each test in ChatGPT - unscalable and tedious
The Solution:
- EarlyAI for automated and AI-based unit test generation
- Seamless integration into IDE and with testing frameworks, no context required
- Covers happy paths and edge cases, providing comprehensive test coverage and insights
The Results:
- 206 tests generated from 9000 lines of code.
- 10% red tests, mostly indicating code issues that needed to be resolved.
- 89% code coverage
- Test creation reduction from 1 month to hours
In this blog, we will see how the contributors and maintainers of open-source ExpressoTS used EarlyAI to create 209 tests from 9000 lines of code in just a few hours with almost 90% code coverage.
For more details and information, and to see a demo of how ExpressoTS leveraged EarlyAI, watch the webinar “How expressoTS uses EarlyAI to increase coverage and quality” with Sharon Barr, CEO and co-founder of EarlyAI, and Richard Zampieri, core contributor and founder of ExpressoTS.
About ExpressoTS
ExpressoTS is an open source Node.js + TypeScript framework designed for creating highly maintainable and scalable server-side applications. Built with flexibility, modularity and extensibility in mind, it enables developers to build efficient applications tailored to their unique needs.
As a growing framework, ExpressoTS relies on four pillars of sustainability: user adoption, documentation, security and robust code coverage. Code coverage is crucial for ensuring reliability.
The Challenge: Time-consuming Test Generation
QA is essential for any software application, but creating and running software tests is an ongoing challenge for any startup. Testing is a consuming effort that requires developer time and resources. In some cases, the organizational culture underestimates the value of QA, and therefore does not invest in frameworks or allocate time for it among developers’ tasks. The inclination towards testing varies among developers as well, with some developers prioritizing other engineering tasks over testing.
These challenges are further exacerbated when it comes to open-source projects like ExpressoTS. This is due to the voluntary nature of the project, making resource scarce and developer time inconsistent. In addition, testing rigor can be difficult to maintain when contributors vary in experience and testing priorities.
ExpressoTS’s core contributors were in search of an automated and AI-powered solution to answer these challenges. Initially, they tried using general-purpose tools like ChatGPT to generate tests. The results were adequate, but they could (and should) have been better.
In addition, testing with ChatGPT took a very long time. The process was manual, requiring them to copy and paste every single function to share the context. Creating each test took approximately 10-15 minutes! This was unscalable, tedious and led to frustration.
The Solution: EarlyAI
ExpressoTS started using EarlyAI as their AI agent for unit test generation. EarlyAI integrates easily into the IDE, generates tests in a single click and derives code context on its own. As a result, EarlyAI allows developers to generate unit tests with minimal effort, automatically handling code context and configuration.
The ExpressoTS team installed the EarlyAI extension directly in their VS Code environment. EarlyAI was able to automatically integrate with Jest, their testing framework, eliminating the need for complex manual configurations.
Once the setup was complete, the team selected the specific methods and functions within the code that needed to be tested. With a simple click, EarlyAI generated unit tests for these selections, automatically understanding the context and with no need to copy and paste code manually.
EarlyAI provided both happy-path tests and edge-case validations, expanding the scope of testing. It helped identify gaps and provide insights, such as ensuring patterns like singletons were implemented correctly.
When the tests were being created, Earl (EarlyAI’s QA Engineer) even provided documentation and suggested code improvements. This further contributed to the open-source team, replacing the need for tedious documentation writing.
With EarlyAI, ExpressoTS has been able to enhance its code quality and coverage without compromising on development speed and while significantly cutting down the time it takes to generate tests.
“EarlyAI demonstrated its ability to handle even complex scenarios with multiple dependencies and interactions, proving to be a powerful ally in software development.” Richard Zampieri, ExpressoTS core contributor.
The Results
With EarlyAI, ExpressoTS’s maintainers were able to achieve:
- 206 tests generated from 9000 lines of code.
- 10% red tests, mostly indicating code issues that needed to be resolved.
- 51 test suites
- 89% code coverage
- Months of developer work saved writing tests - Manually writing these tests would have taken over a month. EarlyAI delivered them in a few hours, allowing the development team to focus on addressing bugs and improvement opportunities instead of writing repetitive tests.
- Confidence in AI for generating tests and peace of mind to focus on reviewing tests or for other tasks.
Deep Dive: How EarlyAI Works
ExpressoTS used Early to automatically generate 206 tests from 9000 lines of code. Here are two examples of such tests:
Use Case #1: The getService Function
The function `getService` dynamically imports a TypeScript (TS Node) module, creates a compiler object and returns it with specific configurations. For this function, ExpressoTS’s expectations were:
- Object Validation: Ensure the returned object matches the expected instance.
- Configuration Accuracy: Validate that the returned object adheres to specific configurations.
EarlyAI was able to immediately analyze the context of the function, generating comprehensive tests. The tests covered comprehensive use cases:
- Validation of the Happy Path: EarlyAI created tests to verify that the function returned the expected instance with the correct configuration.
- Edge-Case Insights: Beyond basic validation, it found potential testing gaps, such as ensuring the singleton pattern was upheld for the class.
Use Case #2: Validating Parameters and Catching Edge Cases
An ExpressoTS’s contributor was working on a function that required specific parameters, one of which was a mandatory string. Initially, the developer did not consider validating for an empty string.
When tests were generated, Early automatically identified this as an edge case. The generated test failed, pointing out that the function was not handling scenarios where an empty string was passed. This prompted the team to add the necessary validation, preventing a potential production issue.
Final Thoughts: Building Better Software with EarlyAI
EarlyAI is more than just a testing tool; it is a platform designed to elevate the way developers approach quality assurance. This is because EarlyAI does not just confirm functionality. Rather, it broadens the scope of what should be tested, prompting developers to think critically about edge cases and hidden bugs. With Early AI, teams can be confident that their code is genuinely robust and ready for deployment.
We helped ExpressoTS in achieving 89% code coverage in hours, not weeks. For teams looking to improve their testing efficiency and code quality, EarlyAI is the tool for you! We support the open-source community and provide a free tier to all open-source projects.
You code, leave the Test to us!