"lolicon" isn't a Japanese term. It was invented by a random American observing things going on in Japan. Then, the term was borrowed, presumably out of convenience. Indirect inspiration by Lolita (even that was a work of fiction) is given too much credit.
Also, honestly, it's not a real term (particularly in the West). It means whatever the person speaking wants it to mean which is not a whole lot. I see it used as a slur against supposed "subhumans" (so being a jerk) or "Cool and feels taboo."