personally going from a 1080p 10yr old higher end Sony LCD to a 4k higher end panasonic I noticed the difference in good and bad ways, the post processing is 'different' and creates weird popping sensations if turned up too high, but the picture is better overall.
Going from a 2yr old 1080p, you'll probably not notice the difference.
A top quality 1080p (or even 720p) TV can look better than a brand new 4K TV especially the cheaper ones.
More pixels doesn't give you better contrast or color. The added pixels require more processing so non 4k material can look awful. You can't even see the additional pixels unless you are very close to the screen.
HDR is a more valuable feature. It gives you wider range of color and brightness but it only works on HDR encoded 4k material.