Tristan Mc Leay - Mutation Testing With Sydtest

Melbourne Compose Group is the monthly in-person meetup for functional programmers in Melbourne, which this month, is on the second Thursday of the month.

Our November meeting will be 6:00-8:00 pm Thurs 12th of November at our regular venue, Activity Room 2 at Kathleen Syme Center in Carlton. Arrive from 5:30 for chat and socialising, talks start at 6:30pm. Please RSVP below.

Tristan Mc Leay - Mutation Testing With Sydtest

Now that use of AI code generation tools has become prevalent, we need a wayto keep our agents honest. Is our code well tested? Have we skipped tests?Are the tests sensible or meaningless? All of these could have been problemsbefore AI, but they take on an increased significance when we can producebigger changes than we might be able to genuinely concentrate on.

Mutation testing is a way of checking our test quality as well as our testcoverage: instead of just asking whether a line ran during testing, it askswhether changes to that line would have caused the tests to fail. In the caseof a typed functional language, that might be swapping an operator, dropping acase, hard-coding a boolean instead of an if or a guard. If the tests continueto pass with a change in place, then the test suite is usually insufficientlysensitive to changes.

Sydtest includes a mutation testing framework for GHC, implemented as a GHCplugin alongside driver code. It was designed precisely for this problem - asour coding switches to AI, we need AI to look at our mutation test results toknow what to do next.

In this presentation, I will take a tour of Sydtest's mutation testing andapply it to a code base. This code base has poor test coverage and was originally hand written but nowadays is essentially agent-coded.

As always, newcomers welcome. Contact Ben on 0407 990 094 if you need assistance finding us on the night.

Hope to see you there :)-Ben Hutchison & John Walker