It's inherent to the medium.
1) As you say, the outer tracks (track 1, 2 etc) are rotating at a greater velocity in relation to the stylus than the inner so the same moment of music is spread over a larger physical area on the outer tracks.
Conceivably (and I can't quite get my head around this) the signal to noise ratio would be more favourable at the outer (first) tracks.
2)The normal pivoting tone arm travels in an arc across the disc and the ideal angle for the stylus (which is usually oval/elliptical) is at the middle tracks. The least ideal at the inner (last) tracks.
3) The effect of spinning a disc under a stylus is to drag the stylus inward toward the centre gravitationally -- thus the bias adjustment on better tone arms to compensate by pressing the arm back out towards the outer rim of the disc. Those forces acting on the stylus affect tracking ability, thus distortion. The effects are greatest at the inner (last) tracks.
4) In many types of music (opera for example)the climax (often the loudest bit) tends to be at or near the end of an album -- thus at the end of the disc where conditions (for the above reasons) are least favourable.
5) At the extremes of the disc the velocity effects will affect the drag the stylus causes. On a system with a small motor this may actually affect speed of playing (though only a pitch perfect listener might notice it).
One or two of these effects can be addressed by using a longer arm. Thus the 12 inch models -- as opposed to the usual 9 inch -- offered by the famous UK brand SME. But this brings with it more mass and the associated problems of stylus compliance and low frequency resonances on warps producing overload at the phono preamp, major voltage demands at the amplifier and pumping of the speaker bass unit.
Some of the geometrical effects can be addressed by employing a radial (straight line tracking)
arm but this has to be driven by a servo which is likely to introduce a sort of hunting effect so that the stylus is constantly at the wrong position within the groove walls. There is also the issue of drag, though some systems have used an optical detector to position the arm.
The fact is that the best sounding analog systems still use pivoting arms with precise but low friction bearings (some a gravity unipivot with just one bearing to reduce friction) of a conventional length and the best mass/rigidity they can achieve.
Solution, certainly for classical music, Compact Disc.