Patent Issued for Software testing in parallel with different database instances (USPTO 11714745): State Farm Mutual Automobile Insurance Company

2023 AUG 23 (NewsRx) -- By a News Reporter-Staff News Editor at Insurance Daily News -- From Alexandria, Virginia, NewsRx journalists report that a patent by the inventors Chauhan, Shaktiraj (Normal, IL, US), Shepherd, Nate (Bloomington, IL, US), filed on May 28, 2021, was published online on August 1, 2023.

The patent’s assignee for patent number 11714745 is State Farm Mutual Automobile Insurance Company (Bloomington, Illinois, United States).

News editors obtained the following quote from the background information supplied by the inventors: “During development of a software application, software developers can prepare tests to verify that the software application operates as expected. Such tests may be associated with unit testing that tests the functionality of a relatively small piece of code, and/or integration testing that tests how multiple pieces of code interact. Because individual tests may be designed to test one or more relatively small aspects of the overall software application, software developers may create numerous tests associated with the entire software application.

“Some tests associated with the software application may build on one another. For instance, a first test may create a record in a database, and a second test may be designed to further access or edit the database record created by the first test. In this situation, a developer may write code for the second test under the assumption that the first test will already have executed and created the record in the database by the time the second test executes. Accordingly, in this situation, the second test may be dependent on data created by the first test.

“In some situations, when a new version of the software application is tested, it can take a relatively long period of time to execute all of the tests and verify that the new version of the software application successfully passes all of the tests. For instance, tests can be written to build on each other in a sequential order as discussed above, and running a large set of tests in a sequence can take a relatively long period of time. As a non-limiting example, tests for a software application may be written in a set of 7000 class files that each include numerous functions, and it may take up to two hours to sequentially execute all of the tests in the 7000 class files.

“The time it takes to sequentially execute a large set of tests can pose challenges and introduce delays during software development, particularly in situations in which multiple developers are working on the same software application. As an example, a team of software developers may generate sixty new builds of the software application in a single day. If each build is tested using a set of 7000 class files that takes up to 120 minutes of server time to execute, the testing may take up to 7200 minutes of server time in a single day.

“As another example, two developers may check out code from a main code branch and independently make changes to the code for the software application. If the first developer submits updated code for testing, it may take two hours to run a full set of test cases on the first developer’s updated code. However, by the time the testing verifies that the first developer’s changes have passed the full set of test cases and can be incorporated back into the main code branch, the second developer may separately have submitted different code changes for testing. The code changes made by the second developer may be incompatible with the changes made by the first developer. Accordingly, even if the second developer’s changes also independently pass the full set of test cases, it may not be possible to incorporate those changes back into the main code branch due to incompatible changes already made by the first developer. If testing of the first developer’s code changes had been performed more quickly, the second developer could have been made aware of the first developer’s changes and accounted for them before the second developer submitted additional code changes for testing.

“Accordingly, it can be desirable to run a full set of tests on updated code for a software application more quickly than described above. One possibility for speeding up the testing process is to run different tests in parallel at the same time. For example, rather than running tests within a full set of 7000 class files in sequence, the tests can be divided into smaller sets that can be run simultaneously in two or more parallel threads. Although executing the full set of tests in sequence may take up to two hours, running different subsets of the tests simultaneously in parallel threads may allow the full set of tests to complete in 30 minutes or less. Developers can thus be notified whether their code changes pass the full set of tests more quickly, and allow code changes that have passed the full set of tests to be merged into a main code branch more quickly. In addition to executing a full set of tests more quickly for a single new build of a software application, running tests in parallel threads can also reduce overall usage of server time when multiple new builds of the software application are tested. For instance, if testing each build of a software application takes up to 120 minutes when tests are executed sequentially in a single thread, it can take up to 7200 minutes of server time to test sixty different builds in a day. If running the tests in parallel reduces the testing time for each build down to 30 minutes as discussed above, testing sixty builds in a day may instead take only 1800 minutes of server time. However, although executing tests in parallel can speed up testing of a software application, in some cases the parallelization itself can cause errors and/or failed tests during testing.

“For example, as discussed above, tests may be written under the assumption that the tests will build on one another in a sequential order. Code changes may pass such tests if the tests are indeed executed sequentially in the originally-intended order. However, the same code changes may not pass the same tests if the tests are executed out of order, particularly if related or dependent tests are executed out of order in different parallel threads, and attempt to access the same database instance.

“As an example, multiple tests may attempt to access the same table of a database. The database may be configured to at least briefly lock the table when the table is accessed by a code element, such that the database table cannot be changed by other code elements while the table is locked. If a set of tests are run sequentially in an intended order, a first test may have finished accessing a database table, and the table can thus be unlocked, by the time a second test later attempts to access the database table. The second test may succeed in this scenario due to the unlocked database table. However, if the tests are instead run in parallel as described above, there is a chance that different tests, executing simultaneously in different parallel threads, may attempt to access the same database table of the same database instance at essentially the same time. This can cause database conflicts, table locking errors, application errors, service errors, and/or failure of one or more of the tests. For instance, if a database table is locked due to a first test accessing the database table, the table may still be locked if a second test, executing in parallel with the first test, also attempts to access the database table. The database table locking can thus cause the second test to fail, even though the second test may have succeeded if it had been executed at a different time when the database table was not locked due to another test executing in parallel.

“As another example, tests may be written such that a first test creates data in a database, and a second test accesses or edits the data that the first test created. If these two tests are executed in sequence, the first test can have successfully created the data in the database by the time the second test attempts to access that data in the database. However, if the tests are instead run in parallel, there is a chance that the second test will execute in a parallel thread earlier than the first test executes in another parallel thread. Accordingly, if the second test executes earlier than the first test due to parallelization, the second test may fail because database data used in the second test has not yet been created by the first test.

“The example systems and methods described herein may be directed toward mitigating or overcoming one or more of the deficiencies described above.”

As a supplement to the background information on this patent, NewsRx correspondents also obtained the inventors’ summary information for this patent: “Described herein are systems and methods for testing a software application using a set of test cases distributed among different instances of the software application that are associated with different database instances. To avoid database conflicts that may occur when different test cases executed in parallel attempt to access the same database, test cases executed in parallel can access different database instances. For example, the use of different database instances can avoid record-locking errors or table-locking errors when different test cases attempt to concurrently access the same record or table in the same database.

“According to a first aspect, a computer-implemented method can include instantiating, by one or more processors of a computing device, a plurality of software application instances. Software application instances of the plurality of software application instances can be executable copies of a software application, and can be configured to execute simultaneously in parallel on the computing device. The method can also include instantiating, by the one or more processors, a plurality of database instances corresponding to the plurality of software application instances. The method can further include distributing, by the one or more processors, a plurality of test cases among a plurality of test sets, the plurality of test sets being associated with the plurality of software application instances. The method can also include executing, by the one or more processors, the plurality of test cases in parallel. Different test sets of the plurality of test sets can execute in association with different software application instances and different database instances. The method can additionally include collecting, by the one or more processors, a plurality of test results associated with the plurality of test cases, and aggregating, by the one or more processors, the plurality of test results.

“According to a second aspect, a computing device can include one or more processors and memory storing computer-executable instructions that, when executed by the one or more processors, cause the one or more computing devices to perform operations. The operations can include identifying a plurality of test cases that test database interactions associated with a software application. The operations can also include instantiating, in the memory, a plurality of database instances associated with corresponding executable copies of the software application, and distributing the plurality of test cases among the corresponding executable copies of the software application. The operations can further include executing a first test set of the plurality of test cases, in association with a first executable copy of the software application and a first database instance of the plurality of database instances. The operations can also include executing a second test set of the plurality of test cases, in association with a second executable copy of the software application and a second database instance of the plurality of database instances, wherein the first test set and the second test set execute in parallel. The operations can additionally include collecting a plurality of test results associated with the plurality of test cases, and aggregating the plurality of test results.

“According to a third aspect, one or more non-transitory computer-readable media can store computer-executable instructions that, when executed by one or more processors, cause the one or more processors to perform operations. The operations can include instantiating a plurality of database instances associated with a plurality of software application instances, and distributing a plurality of test cases among the plurality of software application instances. The operations can also include executing a first test set of the plurality of test cases, in association with a first software application instance and a first database instance of the plurality of database instances. The operations can further include executing a second test set of the plurality of test cases, in association with a second software application instance and a second database instance of the plurality of database instances, wherein the first test set and the second test set execute in parallel. The operations can also include collecting a plurality of test results associated with the plurality of test cases, and aggregating the plurality of test results.”

The claims supplied by the inventors are:

“1. A computer-implemented method, comprising: instantiating, by one or more processors of a computing device, a plurality of software application instances, wherein different software application instances of the plurality of software application instances: are executable copies of a software application that interacts with a database, and are configured to execute simultaneously in parallel on the computing device; instantiating, by the one or more processors, a plurality of database instances, wherein different database instances of the plurality of database instances: are separate instances of the database, correspond to the different software application instances, and are created as in-memory databases by the different software application instances after initiation of the different software application instances; dynamically distributing, by the one or more processors, a plurality of test cases among a plurality of test sets prior to execution of the plurality of test cases, wherein: the plurality of test cases is configured to test interactions between the software application and the database, and different test sets, of the plurality of test sets, correspond with the different software application instances; executing, by the one or more processors, the different test sets in parallel, in association with the different software application instances and the different database instances, wherein execution of the different test sets causes the different database instances to store different data; collecting, by the one or more processors, a plurality of test results associated with the plurality of test cases; and aggregating, by the one or more processors, the plurality of test results.

“2. The computer-implemented method of claim 1, wherein the plurality of database instances comprises record-locking databases.

“3. The computer-implemented method of claim 1, wherein: code of the plurality of test cases is expressed in a set of class files that each include one or more methods, and the one or more processors dynamically distribute the plurality of test cases, among the plurality of test sets prior to the execution of the plurality of test cases, at a class level by assigning different groups of one or more class files, of the set of class files, to the different test sets.

“4. The computer-implemented method of claim 1, further comprising: identifying, by the one or more processors, one or more failed test cases of the plurality of test cases based on execution of the different test sets in parallel; and re-executing, by the one or more processors, the one or more failed test cases, wherein aggregating the plurality of test results comprises determining whether all test cases in the plurality of test cases succeeded during at least one of: execution of the different test sets in parallel, or re-execution of the one or more failed test cases.

“5. The computer-implemented method of claim 1, wherein the plurality of software application instances and the plurality of database instances are instantiated in a plurality of virtual machines executing on the computing device.

“6. The computer-implemented method of claim 1, wherein the different software application instances, of the plurality of software application instances, are each associated with a different processing thread on the computing device.

“7. The computer-implemented method of claim 1, wherein the different software application instances, of the plurality of software application instances, are each associated with a different set of parallel threads on the computing device.

“8. The computer-implemented method of claim 7, further comprising: distributing, by the one or more processors, a group of test cases of a particular test set among a plurality of test subsets, wherein the particular test set and the plurality of test subsets are associated with a particular software application instance and a particular database instance; and executing, by the one or more processors, the group of test cases in parallel, wherein different test subsets of the plurality of test subsets execute: via different parallel threads associated with the particular software application instance, and in association with the particular database instance.

“9. The computer-implemented method of claim 1, wherein: code of the plurality of test cases is expressed in a set of class files that each include one or more methods, and the one or more processors distribute the plurality of test cases, among the plurality of test sets prior to execution of the plurality of test cases, at a method level by assigning different groups of methods, from across the set of class files, to the different test sets.

“10. The computer-implemented method of claim 1, wherein dynamically distributing the plurality of test cases among the plurality of test sets, prior to the execution of the plurality of test cases, comprises: determining predicted execution times associated with the plurality of test cases; and distributing the plurality of test cases, among the plurality of test sets, based on the predicted execution times.

“11. A computing device, comprising: one or more processors; memory storing computer-executable instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising: identifying a plurality of test cases that is configured to test interactions between a software application and a database; instantiating, in the memory, a plurality of database instances, wherein different database instances of the plurality of database instances: are separate instances of the database, correspond with different executable copies of the software application, and are created as in-memory databases in association with the different executable copies of the software application after initiation of the different executable copies of the software application; dynamically distributing the plurality of test cases, among the different executable copies of the software application, prior to execution of the plurality of test cases; executing a first test set of the plurality of test cases, in association with a first executable copy of the software application and a first database instance of the plurality of database instances; executing a second test set of the plurality of test cases, in association with a second executable copy of the software application and a second database instance of the plurality of database instances, wherein: the first test set and the second test set execute in parallel, and execution of the first test set and the second test causes the first database instance and the second database instance to store different data; collecting a plurality of test results associated with the plurality of test cases; and aggregating the plurality of test results.

“12. The computing device of claim 11, wherein the operations further comprise: identifying one or more failed test cases of the plurality of test cases; and re-executing the one or more failed test cases, wherein aggregating the plurality of test results comprises determining whether all test cases in the first test set and the second test set succeeded during at least one of: execution of the first test set and the second test set in parallel, or re-execution of the one or more failed test cases.

“13. The computing device of claim 11, wherein the operations further comprise: creating a plurality of virtual machines on the computing device; and associating the different executable copies of the software application with different virtual machines of the plurality of virtual machines, wherein the different executable copies of the software application execute in parallel in the different virtual machines.

“14. The computing device of claim 11, wherein the operations further comprise: distributing individual test cases of the first test set among a plurality of test subsets associated with the first database instance; and executing different test subsets of the plurality of test subsets via different parallel threads associated with the first executable copy of the software application and in association with the first database instance.

“15. The computing device of claim 14, wherein: code of the plurality of test cases is expressed in a set of class files that each include one or more methods, the plurality of test cases is dynamically distributed among the different executable copies of the software application at a class level, by: assigning one or more first class files, of the set of class files, to the first test set, and assigning one or more second class files, of the set of class files, to the second test set, and the individual test cases of the first test set are distributed among the plurality of test subsets at one or more of the class level or a method level, by assigning different subsets of methods of the one or more first class files among the plurality of test subsets.”

There are additional claims. Please visit full patent to read further.

For additional information on this patent, see: Chauhan, Shaktiraj. Software testing in parallel with different database instances. U.S. Patent Number 11714745, filed May 28, 2021, and published online on August 1, 2023. Patent URL (for desktop use only): https://ppubs.uspto.gov/pubwebapp/external.html?q=(11714745)&db=USPAT&type=ids

(Our reports deliver fact-based news of research and discoveries from around the world.)