There's nothing wrong with the form factor. I love the size of the iPhone SE and I have been extremely happy to be able to essentially keep the same phone shape since the iPhone 6 since I pretty much feel it was/is the perfect size and any bigger is a huge inconvenience for me. The problem is that they released a phone 2 years later that is not any real type of upgrade. I'm supposed to shell out $450-$600 for crappy 5G, a minor processor upgrade and allegedly slightly better battery life? No thanks.
The form factor isn't the problem here. The problem is releasing the exact same phone and expecting people to pay full price for it. The whole appeal of the refreshed SE was accessible pricing so I assume most people did like me and just purchased it outright instead of leasing. With no option to trade in and continue/renew a lease, why would any of us want to let go of a big chunk of cash for such pitiful upgrades? Same camera, same guts, no noticeable performance change, you only get $150 for an SE 2020 trade-in.... what's the appeal?
I got an iPhone 2022 SE for my S/O and it's perfect since it's replacing a 2016 SE but I had no desire to upgrade my 2020 SE because it felt like a waste of money. It feels like we have the same phone now. I don't even envy the 5G connectivity since 5G still pretty much sucks and the phone seems to be more reliable if you disable it and stick with LTE.