Running Tests in Multiple Programming Languages

One of the major benefits of writing libraries in progsbase is the ability to translate the code into most other programming languages. To make sure that the code runs the same in each programming language, you can use the progsbase tester tool to run the library's test suite in all languages at once.

Making your Code Available for the Test Tool

To make your library available for the tester tool, upload the code to the progsbase repository using the progsbase tool. Each library is known by its namespace, name and version. These three pieces of information are given to the progsbase tester tool to run the library's test suite.

Progsbase Test Suite

To make a progsbase test suite, make a function named test in a file called test in the test source directory. This function takes no input and returns a number. This number is the number of failures. In other words, 0 is success and any other positive number means failure.

For an example, check out this function.

https://repo.progsbase.com/repoviewer/no.inductive.idea10.programs/datetime/0.1.17/test/test/test/

Progsbase Tester Tool

The progsbase tester tool is an online service with many programming languages installed. You can ask the online tool to run your library's test suite by visiting the library page on the progsbase repository. Take for example, the library Fractions v0.1.6 in the namespace no.inductive.libraries. You can access the library page here:

https://repo.progsbase.com/repoviewer/no.inductive.libraries/Fractions/0.1.6/

On this page you will see a button "Run tests for all languages". (If you do not see this button, force a refresh with for example Ctrl + F5.)

Pressing this button will make the tests run. After some time, you should see a table such as the following:

Here we can see a list of languages and whether the test suite of the library succeeded or failed in each language. We can also see the number of failed tests and the time taken to build the library and run the tests.

In this list, T-Form means a version of Java where the expressions are expanded. Assembler is the progsbase assembly code generator. The two versions of C are 89 and 99, as can be seen by hovering over each.

Notice here that the interpreted languages have the fastest iteration times for building and running the tests, while C# and Visual Basic.Net spent a lot of time in the loading stages of the dotnet tools.

If we look at a more heavy test suite, we see that JavaScript is the fastest followed by C and C++. The reason JavaScript is the fastest is that it is optimized for a language similar to progsbase. C and C++ are fast because they are compiled languages.

https://repo.progsbase.com/repoviewer/no.inductive.libraries/Accrual/0.1.9/

The test suite might fail for some of the languages. In that case, there are two explanations: Either the program did not follow the programming rules required by progsbase, or there is a bug in the writer for that language. This can be seen in, for example, version 0.1.25 of the PNG library:

https://repo.progsbase.com/repoviewer/no.inductive.libraries/PNG/0.1.25/

Progsbase Tester Tool -- Command Line

You can also run the tests from the command line using the following command:

curl -s https://tester1.progsbase.com/no.inductive.libraries/Fractions/0.1.6/

This will produce the following output:

[
  {"success": true, "failures": 0, "language": "Java", "version": "5", "time": 2.972},
  {"success": true, "failures": 0, "language": "Java (T-form)", "version": "5", "time": 2.452},
  {"success": true, "failures": 0, "language": "Assembler", "version": "x86-64", "time": 0.905},
  {"success": true, "failures": 0, "language": "C", "version": "89", "time": 1.195},
  {"success": true, "failures": 0, "language": "C", "version": "99", "time": 0.999},
  {"success": true, "failures": 0, "language": "C++", "version": "98", "time": 1.115},
  {"success": true, "failures": 0, "language": "JavaScript", "version": "7", "time": 0.367},
  {"success": true, "failures": 0, "language": "TypeScript", "version": "5.4", "time": 6.668},
  {"success": true, "failures": 0, "language": "C#", "version": "8", "time": 12.249},
  {"success": true, "failures": 0, "language": "PHP", "version": "8", "time": 0.178},
  {"success": true, "failures": 0, "language": "Python", "version": "3", "time": 0.188},
  {"success": true, "failures": 0, "language": "Visual Basic", "version": "12", "time": 9.006},
  {"success": true, "failures": 0, "language": "Swift", "version": "5.10", "time": 6.329},
  {"success": true, "failures": 0, "language": "Ruby", "version": "3", "time": 0.396}
]

Releases and Snapshots

When running the tests for a release version of a library, the results are cached: The tests are guaranteed to be the same each time it is run.

If you run the test suite of a snapshot release, then the test suite will be run again each time. The reason is that the code might have changed since last time it was run, so when developing and debugging, use a snapshot release.