Description

The encVerify↗ function enforces that expRHat has an even y-coordinate instead of enforcing that R has an even y-coordinate. The y-coordinate of R must be even so that the adaptor signature decrypts to a valid BIP-340 signature (which requires even R for nonmalleability).

// fail if expected R'.y is odd
if expRHat.Y.IsOdd() {
    return fmt.Errorf("expected R'.y is odd")
}

Impact

Since encVerify does not enforce that R has an even y-coordinate, it will consider both (R+T, e*d+k) and (-R+T, e*d-k) to be valid adaptor signatures, but only one of them decrypts to a valid BIP-340 signature. If a covenant emulation committee member generates adaptor signatures with odd R.Y values, the adaptor signatures will be valid according to the handler for MsgAddCovenantSigs, but they will not decrypt to signatures accepted by Bitcoin, preventing slashing from occurring. Note that encSign currently does correctly generate even R.Y values; this is only an issue with encVerify.

Additionally, since EncSign keeps generating nonces until a signature verifies (which is a correct way to handle rarer failure conditions), incorrectly rejecting signatures with odd RHat.Y values (which happens half the time) causes a geometrically distributed number of iterations for signing instead of a practically constant number of iterations, decreasing signing performance.

Recommendations

Enforce that R.Y is even instead of enforcing that expRHat.Y is even in encVerify:

    expRHat.ToAffine()

-   // fail if expected R'.y is odd
-   if expRHat.Y.IsOdd() {
-           return fmt.Errorf("expected R'.y is odd")
+   // fail if R.y is odd
+   if R.Y.IsOdd() {
+           return fmt.Errorf("R.y is odd")
    }

    // ensure R' is same as the expected R' = s'*G - e*P
    if !expRHat.X.Equals(&RHat.X) {
        return fmt.Errorf("expected R' = s'*G - e*P is different from the actual R'")
    }

Remediation

This issue has been acknowledged by Babylon Labs, and a fix was implemented in commit 714b8ef9↗.