Saturday, December 08, 2007
Just a Reminder...Robots!
Look folks, I've asked before and it doesn't seem to do any good. Please don't run automatic software that checks my site a zillion times per day for updates. It runs up my bandwidth and causes my webmaster to chew my ears off. It's a waste of everyone's time to try to figure out who the perps are to either contact them or block them, whatever. I understand there's stuff out there that makes it very easy to do this especially for Mac users. I'm asking folks to please refrain so that I don't have to go through all the nightmare and expense of getting a dedicated server to handle the traffic. Thanks!
I looked at your site, and everything that interested me was sold out. I suggest you turn off your site for a few days put a message saying you were all out of stock anyways, and put a message to the robots stating about your blogs RSS, and you will post about all inventory updates in there (anyone who can run a robot knows what rss is). State that its a last ditch effort and you will just shut down the website if it doesn't end. Or get a new blog and post just inventory updates in it. Most likely the robot users don't read your blog. After that add something like INVENTORY RSS Feed on your main page in a GIANT button so new people won't start using robots as well.
ReplyDeletePart of the fun of being in the Atwood Nation is "hunting" for the tools he's going to release. you know where there going to be and an idea what time, but its just luck in the end. If we hunted with robots there would be no more game and that would not be good or FUN.
ReplyDeletei would have liked at termite, bottlebug, G2 prybaby ect. really any one of his creations.
i looked before i went to sleep and when i woke up. theres next time. STOP the BS he is a fair guy and is wasting his time with looking for automatic GREEDY people. He could have made another few tools with his time. The more Idiots = less tools and idea's.
SO STOP! thanks for your time
This morning about 10am I checked on the termite because the header above the picture said "In stock". The item details said "Coming soon". I returned to the site an hour later to find the item sold out. Maybe the item was selling out when I checked the first time, but it sure was fustrating!
ReplyDeleteBTW, great products...and I always look forward to new blog entries.
Peter,
ReplyDeleteHave you or your webmaster looked into restricting these robots from your webpage?
http://www.javascriptkit.com/howto/htaccess.shtml
We have blocked people in the past and we will continue to do so. My policy is clear, I won't tolerate robots or other kinds of automatic site checking software that runs up my bandwidth. If I think you are running this type of thing and you consistently show up with an order moments after I post a new item I reserve the right to cancel your transaction and refund your money. This is a fairness issue and there are many people out there who email me complaining that they are never able to get a single piece of my work. Meanwhile the robot people sweep in and grab pieces left and right. It needs to stop.
ReplyDeletePoint of clarification here.
ReplyDeleteWhen you say "robots", do you mean using Google Reader for this blog or something else entirely?
Thanks!
I'm talking about people setting up automatic software to check atwoodknives.com not this blog.
ReplyDeleteSome of the excessive traffic on my site may be due to people refreshing the pages constantly by hand but with a setting in their browser that pulls all the images. I'm no web expert, just going by what I am told but my webmaster has complained endlessly about a few people who are running up huge visit and transfer stats.
Guilty as charged...
ReplyDeleteI am one of those people who kept refreshing too much and ended up eating excessive bandwidth. Problem is, I haven't the faintest idea about how many accesses I can do during the day in order to visit the site in a "fai" way :/
F.
unfortunately, on the internet, calls for fairness and decency are usually unheard by those who most need to hear them.
ReplyDeleteJust use technical means: more than 10 accesses to the same page in 10 minutes? blocked for 15. blocked more than 5 times in one day? perma-blocked.
adjust as necessary. if your webmaster has the ability to discern that these abusive requests are coming from particular users, then it should be trivial to block them. block all anonymizing proxy subnets while you're at it, and you might fix your knock-off problem too. ;)
Hi...I have yet to purchase a tool from the website (mine are from the pricey secondary market), and I have no idea how a robot works, so this is not really my issue...but...I do know that getting people to voluntarily stop a successful method of obtaining rare items will be difficult, particularly if they are not blog readers and don't even see your appeal. If the problem is bandwidth, had you considered making the page that specifies availability, to have no photos? A very sparse page, something like Google's main page. Then you can offer links to photos if they are desired by the customer. If nothing else this would help keep your bandwidth way down. --Bolster (on EDC forums).
ReplyDeletePS: While I'm here, I am happy to report I snagged 2 Mini Keytons. One went immediately to my contractor, the other on my keychain; very useful for pulling staples and opening cans while we refurb our new house. Thanks a million! I know most people collect these, but they are very USEFUL, too!
Back to the future... I just started using a "links" text-only browser for accessing the Atwoodknives home page.
ReplyDeleteHadn't been using a lynx-like web client for YEARS! A nice jump into the past...
This should dramatically reduce bandwidth consumption... and save Peter from his webmaster's wrath :)
Francesco
I don't use a robot, but I sure can see why someone would. I've checked this site several times during the course of a couple of days. I would love to buy something, e.g. the new mini gasbaby, but by the time I even see them, they're sold out.
ReplyDeleteJust a suggestion: why don't you post everything on ebay. You'll make more money; people will actually get a chance to buy something & the site robots should go away....
I have another suggestion: Create a product-availability page with no images.
ReplyDeleteIf it's just text, those images won't be downloaded every time someone checks the page, and bandwidth should be significantly reduced.
Just to reiterate what others have already said, this is EXACTLY the problem that rss is meant to solve. It would take some modification to your web site, but using an rss feed to announce your new products would allow anyone with an rss reader to follow your releases and would replace your relatively heavy main page with a very lightweight file that you wouldn't mind having people reload every 15 minutes. You could also have a very lightweight, text-only html page for others who prefer using a web browser (or bot) to keep tabs on your stock, as someone else suggested.
ReplyDeleteIt's not that I think the "bad guys" should win, but I agree with the commenter who said you will never be able to stop all users from using automatic means of checking your site. In fact, many would argue that the web was designed explicitly to allow such activity.
Philosophies aside, it is impractical to try to ban any bot user. Consider, for instance, that 1000 users who work for the same employer may be behind one ip address. When you ban the address, you ban all of those users. In addition, since most home internet users get assigned dynamic ip addresses by their ISPs, getting a new ip address can be as simple as cycling power on their router. Finally, using web proxies like TOR allows web users to "appear" to be using a different address than their true one. In short, banning ip addresses is a bad way to deal with the problems that you are experiencing and is more likely to cause problems for innocent bystanders than for the real "bad guys."
I can appreciate the desire to keep your business on a human scale and that part of that goal is making seller-customer interactions as "real" as possible. Unfortunately, you will never be able to convince your users to stop using bots and banning them is impractical. I really recommend looking at rss as a means of reducing the bandwidth required dramatically.
PS-It occurs to me that you might also want to think about email notifications. Look at how Thinkgeek.com (for instance) puts a button next to each item where a user can enter their email address to be notified when a particular item is available. If your email notifications went out 6 hours before updates were posted to your site, that would make web bots sort of useless, no?