You are working for money, the dirty money, not your boss

you don't have to tell him how many hours you worked on your job on weekend, you just tell him I have got the job done.He doesn't pay you overtime and he doesn't mind how hard you worked.

Why you worried about his feeling or his saying?

You get paid by the hours you stay in your office, be there and find a way to please yourself.