WordPress scraping

Discussion in 'Digital Marketing' started by GreenUtah, Apr 18, 2013.

  1. GreenUtah

    GreenUtah LawnSite Senior Member
    from SLC, UT
    Messages: 866

    I was looking at my error file, which is not big since the site is technically not finished and we're not actively promoting it yet. Low and behold, it looks like the bots are looking for variations on an admin page for WordPress. No WP content or index pages.. specifically admin. Since I don't use WP and have no plans to, they are just sucking wind, but with all the news lately, this is interesting with a brand new site that isn't listed anywhere. That's fun.

  2. greg8872

    greg8872 LawnSite Senior Member
    Messages: 300

    Your site is listed here, which has high visibility, get used to it, you will have tons of bots trying to visit ya, nothing to worry about usually.
  3. GreenUtah

    GreenUtah LawnSite Senior Member
    from SLC, UT
    Messages: 866

    yeah, I know it's listed here..once..lol..and it's showing on first page of google for a search specific to my domain name. I just found it interesting that those 3 specific terms were the keywords being entered/searched for and not content related WP pages (which there were zero entries for) I'll even write one of them as you while you were eyeballing the page perhaps, since I know WP is in your wheelhouse, but that certainly looks like a pattern to me when only variations of the robot.txt (which I also haven't put up) and a couple of mis-pointed internal links were the only other things in this log. (it is from the error log)
  4. greg8872

    greg8872 LawnSite Senior Member
    Messages: 300

    Nope, it wouldn't be me or anyone else familiar with how sites are built. Simply looking at the source code for the page will tell you if you are using WP.

    You have to remember, there are numerous sources out there that give lists of recently registered domain names, and there are tons of systems out there designed to scan those new domains. So it most likely a bot trying to find new WP installs, that are most likely not yet secured, or just to index later for once a vulnerability is published for the current version it finds...

    It is quite easy to write a system like that, I know. (Actually, for me, it was a fun puzzle to figure out how to write. I'm just that much of a geek)

  5. Joe Shooner

    Joe Shooner LawnSite Member
    Messages: 187

  6. Turf Helpers

    Turf Helpers LawnSite Member
    Messages: 25

    I agree with Joe. There's absolutley no legitimate reason for a search spider to be crawling for an admin panel unless it was trying to hack it. Legitimate engines will only seek pages through links (through your website or others). A spider just randomly trying to access pages that don't exist sends up all kinds of red flags!
  7. tonygreek

    tonygreek LawnSite Gold Member
    Messages: 3,915

    Happens every day. It's only alarming if you're not following best practices.
    Last edited: May 10, 2013

Share This Page