Offensive because it’s putting the blame on ‘you’ for his crap, but still…
There’s something to this.
If you’re overweight, you may not have ‘let yourself go’ in the way that means you are so satisfied with your relationship that you take it for granted — but you might be depressed and unhappy (whichever came first, the depression or the weight is rather like the chicken & the egg) which will means you likely aren’t being the best partner.
Taking care of yourself is the most important thing you can do for your relationship — there can’t be a happy two of you when one of you is miserable.
Not that this excuses his horrible behavior (or yours, should the situation be reversed); a commitment is a commitment. To forgive him or not is a separate issue, but in any case, together or apart, you must begin taking care of yourself. That includes your weight, your depression, and your unhappiness.