There seems to be a problem with the Tic Tac Toe game. It will set your affection to 0 when you play that minigame, there doesn't seem to be a reason for it to do so.

X = Players Affection .
Y = Load the amount you already have.
Z = Get more from winning etc.

I'm working on figuring out a solution to this problem, but for now, try not to play Tic Tac Toe in the game.