Not being a Nuclear Physicist, I'm just pulling this out of the air, but I would imagine that changes to the fine structure constant would change the relative ratios of fusion and fission products in a star, so one should look at the spectral signatures of these binaries and look for small differences in the emission spectra of key nucleotides.

Trouble is, as you note, the lifetime of the more massive member becomes shorter rather rapidly as it becomes more massive, so there may be little time for these differences to accumulate.

Perhaps if one studied similar mass-ratio pairs, with different orbital distances you'd see some effect (but you'd have to correct for relativistic effects which also vary in these situations - close orbit to a massive primary).

There may be more scope for study over long timeframes in situations where a "normal" star is orbiting a compact object and the evolution can be studied to longer ages, but of course one must correct for whatever happened in the original collapse of the massive member of the binary, which probably had a substantial effect on its companion (loss of surface layers, intense neutron and neutrino irradiation, shot-blasting with debris etc...