I can’t speak to GitHub or Medium, but the Wikipedia plugin still works perfectly for me. When you say sites return “Disallowed”, is that in the error column under the Log tab? (I don’t think I’ve ever seen that error, which doesn’t help with figuring out what’s going on here.) Just to eliminate the obvious, does switching the Settings between Search and Crawl make any difference?
↧