Depends, what sort of fertiliser was it? If it was an ammonia based fertiliser, probably not.
Tip: Sprinkle a small amount of Sulphate of Ammonia onto a dandelion crown in a lawn, and burn it clean out! You'll have to re-seed, but the dandy will be gone!
What you'll probably have to do is scarify it a bit, then re-seed, otherwise you run the risk of weed seeds or moss taking over the affected area.