Anansi's little experiment on Ponyville was a major turning point for him. Not only did he learn more about his abilities but also why so few villains had ever succeeded in conquering Equestria despite how foolishly trusting the ponies are.

Because there is a always alicorn that can and will stop them.

Only this time, the alicorn is not trying to stop the bad guy out of a desire to save others. No, this alicorn wants what was taken from her and has no concern over how small the bad guy actually is. Having lost the foundation of her character, Twilight is a mess, her thoughts erratic and impulsive. She needs to find what she lost or else she might go crazy.

Unlike all the other times she fought against villains, this was personal.

And Anansi knows this, hence why he runs despite having lived in the Everfree for all his life. From this day on, to the day he returns to the ether, his one fear will be this purple pony, chasing him through the woods.