Welcome to
The DFN Weekly

I was unable to write the light-hearted essay I originally had in mind, until I realized that part of our strength is in continuing to move forward. I trust that you will accept my humor as originally intended, and in full acknowledgment of recent events. ..OT

Gullible

By

Old Tom

OT Scripts

Did you know that the word gullible is not in the dictionary? Even Voltar has a dictionary - please! Be surprised, and take a look. Do you see what I mean; were you surprised?

I have my dictionary open as I type this, and I just scanned the words from gull to Gulliver. I hope you do too, or you'll miss my point.

Did you know that few things on this planet are more gullible than your web server? Your web server believes precisely what it is told, with no outside checking or verification. How do people steal your bandwidth? By lying to your server.

Let me explain.

As you go surfing the web, your browser acts as your agent. That is, it acts upon your requests. For example, you click on a link promising the best set of whatevers on the planet. Your browser, acting as your agent, pops on over to whatevers.com, collects the page of whatevers, and displays it on your screen.

Because of its purpose in collecting and displaying whatevers on your behalf, your browser is technically known as a User Agent. You may have noticed references to "user agents" in your web stats. You may have noticed Mozilla (Netscape), IE, and various spiders listed. A User Agent, then, is simply a piece of software that acts like a browser.

I'm sure your browser is just like mine: Honest and forthright. Never would it lead you astray, and most certainly it cannot possibly concieve of lying to your server. Browsers do not have hidden agendas. Your browser is acting as your agent, and therefore its only concern is to carry out your wishes as completely as possible.

Have you ever seen those legal disclaimers saying so-and-so is "not acting as your agent"? You sign something acknowledging that so-and-so is acting on behalf of the bank, or the real estate office, or the other guy signing the contract. So-and-so is not your agent; so-and-so is their agent.

What I'm getting at is this. The User Agent is just that - the user's agent. It is not honest and forthright with your best interests at heart. Oh, no, not hardly! It is working on behalf of that particular user. When you are the surfer, this is a good thing. Your browser will happily suck down as much bandwidth as it can - for you. That, of course, is an agent's purpose in life.

Not all agents are honest and forthright. Not all agents abide by your rules of engagement. Not all agents acknowledge that there are rules of engagement.

Again, let me explain.

How does a browser request a web page? And, why should you care? The second question is easy to answer: If you're paying for bandwidth, you care. You can better protect yourself if you better understand how you're getting ripped off in the first place. The problem, remember, is that browsers can and do lie, and that servers can and do believe everything the browser tells it. It is truly difficult to concieve of anything more gullible on the planet. The server wants to believe the lie, and is designed to believe the lie. "I'll still respect you in the morning" is happening a hundred times a second.

When a user agent (i.e., a browser) requests a web page, it rings up the server, and presents the request. By way of introduction, the user's agent tells the server a little bit about himself. The server takes the attention at face value; the server truly doesn't care whether the introduction is or even could be true. The server, being as honorable as she is gullible, provides an accurate introduction in return. Once the introductions are complete, the server provides the requested web page, which the user's agent in turn displays to the user.

This "introductory information" is what we use when designing our htaccess files (assuming our server is Apache). How does the server know the referring url? The agent's identity? Origin? Cookie? All this is supplied by the user's agent, and it can be all lies and fabrication. All information is supplied by the user's agent. The server does not check with her friends. If the agent said it is so, it must be so.

As you know, we generally protect ourselves from hotlinking, by checking the referring url. If the referring url is correct, we allow things to proceed. A sneaky user agent, of course, knows what that referring url should be, and therefore tells the server precisely what she wants to hear.

What I have just described is better known as a "site sucker" or "offline browser". That's how they work. That is, they work by lying to your overly-gullible browser software.

How do you (or your server) become less gullible? By doing a bit more checking around. Many of the less-than-desirable users' agents identify themselves as part of their introductory information. Therefore, you can see their names in the "User Agent" section of your web stats. And, therefore, you can deny access based on the user agent's name.

Let me assure you that there are browsers out there, which will be so very happy to keep that cute gullible little server of yours, very busy indeed.