It depends

J2's in my school didn't pay federal tax when I was a postdoc because school HR didn't deducted it from thair pay-checks. It is hard to claim it from IRS after you have paid. Treaty doesn't state very clear.