Differential fuzz testing

Differential fuzz testing is a form of fuzzing where the effects of the execution of two (or more) programs are compared to detect differences. During the course of this engagement, we started developing a harness to perform this kind of testing. Even though we were not able to complete the harness due to time constraints, we shared our work with SyncSwap Labs as a basis on which they could build a comprehensive test. The current version of the harness already detects some differences in behavior in the mint / add_liquidity functions, which given the same inputs, do not seem to match in behavior in at least two ways: there are cases where SyncSwap reverts while Curve does not. Additionally, the amount of LP tokens minted to the user is often slightly different, with the SyncSwap codebase having a tendency to mint a lesser amount in absolute terms. We could not identify the root cause of these differences with certainty. While these inconsistencies do not necessarily represent an issue, we strongly advise SyncSwap Labs to identify and document the causes of the differences in behavior.

In order to set up the fuzz harness, we set up a Foundry test, which ultimately instantiates one SyncSwap Crypto pool and one Curve pool. Instantiating the pools requires deployment of other helper infrastructure, including factory contracts, mock ERC-20 tokens, math library contracts, and so on. To make meaningful comparisons, the two pools need to be deployed using the same curve parameters. Additionally, we removed fees by making targeted modifications to the code of the pools, which was otherwise left unmodified.

Deploying the Curve code required identifying how to obtain the corresponding bytecode. This can be done by manually compiling the code and embedding it as a hexadecimal byte string in the source as well as via the use of FFI cheat codes, which allow to invoke the Vyper compiler at test runtime.

The harness is engineered to contain one fuzzable entry point for each feature exposed by the contract — the primary features being minting and burning of LP tokens and swapping. The harness performs the required operations to do the same operation on both contracts; this entails, for example in the case of minting/adding liquidity, minting the required tokens to be added as liquidity, transferring them or granting the needed approval, and invoking the SyncSwap and Curve contracts. After the operation is performed, the execution results and contract states are compared to detect differences.