It takes more than 3 month for a COS. So if the current F1's OPT gonna expire within 3 month, you better file the F1-F2 along with F2-F1.
You can also file F1-F2 after your spouse's F2-F1 gets approved. You have 2 month grace period after your OPT. But there are some risks if you file the COE during grace period.