asks for ecryptfs password on apt-get upgrade

Often when I apt-get upgrade, I get a request for my ecryptfs passphrase. Often several times in a row.

I have to run ecryptfs-unwrap-passphrase every time to get the passphrase. Kind of aoying.

Is there any way to avoid this (apart from not having an encrypted home folder)?

I'm using XUbuntu 16.04.

Thanks in advance,
Karol.