That Not So Fresh Feeling

My company is at the beginning of a what will end up being a fairly long, exhaustive migration process. Probably on the order of 12-16 months, migrating web sites from a set of servers on one side of the country to a set of servers on the other side. It’s not your typical forklift migration (where you actually move the servers and plug them in at their new home); instead, it’s literally moving files, mail, DNS, etc. to a new platform.

It’s pretty daunting, pretty complicated, and can occasionally be pretty cool.

On the flip side, it’s now 2:55AM Eastern in Boston (where I started my day), but I’m in Phoenix where it’s actually only 11:55PM. That’s a sign that maybe things didn’t go quite as smoothly as one would have hoped.

The step we’re on is a step where we take over DNS for folks. It’s always somewhat difficult, because we’ll get a big list of domains and have to figure out whose record (our nameserver’s or the other nameserver’s) is the “real” record. It’s not generally too tough to figure it out (you can judge by the SOA of the records) and the number of domains is usually short of 100k, so as long as you’re accurate to within 1-2%, it’s not too bad. That’s 1000 guys who might break, which is pretty easy to handle with a good support team and some quick script fixes.

Let me take a step back. The process is actually that our nameservers need to become the authoritative nameservers for the domains we’re moving. This allows us to later change their DNS to point to their new home, and it all kinda works. We have to get the domains, merge them into our nameservers, become authoritative, and then fix what breaks.

We did that yesterday. Except it wasn’t 100k domains. It was 1.2 million. And the domains weren’t coming from a single, well-maintained nameserver. They domains came from three, somewhat munged together nameservers. There were internal conflicts, conflicts with our servers, missing zones. A host of issues. We thought we’d worked most of them out and gotten the problems down to, at most, 4-5k domains. That’s a lot, but in reality, it’s less than 0.5% of the total domains.

“Pretty good,” you say.

“Not quite,” I say.

For you see, there weren’t just three nameservers. There were five. So there’s a couple thousand domains we missed. And we also missed some of the conflicts (either by omission or by grabbing the wrong data). In the end, it was closer to 12k domains that were wonky.

That’s still only 1%. Damn good, given all of the variables.

Except 12000 broken domains leads to a whole lot of phone calls and emails. And some angry customers. And some tired folks staying up to fix things that they weren’t responsible for breaking. And one tired folk–me–staying up because he feels guilty for only being 99% good enough.

DNS is a fickle beast. Thankfully, it’s pretty quickly fixable. Once we’d identified some global problems, we could fix them rapidly and put big chunks of the broken domains back in working order.

I often argue with people who think the “Chinese Market” is a valid business plan. You know, the folks who say “hey, if we can just get our product in front of 100 million people, and get 1% of those people to buy, we’ll be rich!” Except, of course, it doesn’t really work that way. It’s hard to get a product in front of that many people who would be interested in buying, and it’s hard to get 1% of any audience to buy anything.

Well, not in business plans, at least. It does work that way in technical issues. If you’ve got a huge enough base of users, the smallest mistakes can have a big impact on your company and team. In these cases, sometimes being 99% accurate isn’t good enough.

Here’s a graphical representation:

You see, with 100k domains, you never quite reach screwed. It’s manageable.

With 1.2 million domains, you’re pretty much totally screwed.

I poked over to Slashdot this morning before I ran off to basketball. I saw this:

Holy crap. That’s ugly.

Clicking the little link to vote takes you to a Thank You screen that says that voting will help ensure the best stories are presented. I would have expected something more interactive, more (dare I say it) Web 2.0-y.

Then I actually read the Thank You message:

“Thank You for participating in the Slashdottit Rating System

Your vote in the Slashdottit system will help insure the best stories are presented to our readers, with hyper accurate numbers to indicate their relevance and general awesomeness. Only by requiring the contribution of every random user of the internet can we guarantee the most scientifically perfect numbers will be generated. These numbers will guarantee that every story that appears on Slashdot will be interesting, insightful, and flawless. Vote early, vote often.”

Ha. Clever. April Fool’s. Well played.

I’ve been reading a lot of search engine stuff in my feed reader recently. I used to be deep into the search engine optimization knowledge, but at some point, I realized that it was, at some level, just scummy. Not the idea that you’d understand how engines work and do the little things to make your site rank appropriately. No, it was the other stuff, like link exchanges and link buying and the general dishonesty that comes along with that. When I go to a search engine, I want to actually find what I’m looking for, not have to dig through a bunch of crappy sites that think they deserve my traffic.

It got worse when AdSense came along, and it got even worse as Digg, Facebook, MySpace, and the other social networking-type sites got big. Now, not only were people gaming the engines, they were throwing up lame articles and gaming other systems to get both the search juice and the traffic. Their spammy site gets the best of both worlds, and the rest of us deal with more spam–just not of the email variety.

This week was a big to-do about one of these SEO/SMO guys who got banned from now-Yahoo! owned blog widget because he was posting how to hack it (and, quite frankly, being an all-around douche). So, a guy who games the system for a living was bitching about being banned from a free tool that he’d been posting how to hack. Topping it off, a whole bunch of other SEO folks (many of whom I’ve been reading for a few years now) hopped on and defended the guy.

I just don’t get it.

I understand that the whole idea behind this widget (MyBlogLog) and behind other sites (like Digg, Flickr, etc.) is community. You build a community and you get more than just the functionality of the widget, you get the benefit/fun of the community. It’s all so Webtwopointohy.

Finally, a voice of reason came through my feed reader. I’m hoping we’re reaching a tipping point. I’m hoping we’re reaching the point where every sales and marketing guy out there looking to score some quick money doesn’t look at every new site and widget as something to game and make money. Now, I’m not against making money. I’d love to create a site that has some value to people and figure out a way (ads or not) to make some money. But the group of folks who exist solely to put up a site with ads, get it on Digg, and get enough sheep to click on it need to go away. They used to be called spammers, and it’s about time we go back to calling them that.

Remember back when I mentioned that I should build a little centralized social networking manager where you could manage your profile in one place?

Well, I don’t have to anymore. ProfileLinker did it for me.

TechCrunch pointed to them today, which means they’re bound to get a good bit of traffic today. Reading through the comments, folks are hitting on many of the reasons I never got energized to actually do anything about it: the networks could cut you off; users would have to really trust you to give you their login/pass; there are so many networks that you’d be constantly trying to keep up as new networks emerged (as well as keep old networks working).

Partnering with the networks is the only way to make it work long term, I think, and that’s a tall order given that any centralized management system removes page views from their site, thus removing ad views from their site, thus directly reducing their revenue stream.

Still, it’s an idea that needs to happen because managing multiple profiles is just silly.

In part one, I told you what to do if you just want an email address or a blog. In part two, I’ll tell you want to do if you fall into the category of folks who just want their own little nook on the web.

I want my own small little corner of the internet!

Awesome! I’m going to assume you’ve got a domain name (how else can you have your own cool little place without owning www.coffeepotamus.com?). What you need now is web hosting.

Actually, what you need now is an idea of what you’re going to do with your site. Maybe you’ll have a blog and some images of your friend and family? (Using friend rather than friends was unintentional, but funny, so I decided to not correct it.) Maybe you want to throw up those mp3s of your band that you’ve kept squirreled away on your hard drive.

Or maybe you want to go full bore Flash with games and forums and a whole site devoted to the awesomeness that is you.

In any case, you need to make a couple of estimates: how much disk space you’ll need and how much bandwidth you need. For a lot of folks, this is the toughest part of getting a website as it’s the largest way web hosting companies differentiate themselves from each other.

But … I’m going to let you in on a little secret.

It doesn’t matter.

These days, almost any plan that runs you $7-8/month is going to come with gigabytes of diskspace and tens of gigabytes of bandwidth. Guess what? Unless you’re really really popular, you won’t even use a tenth of that. Most people won’t even use 1/100th of that.

The lesson here? Pick a hosting company based on things that will have a bigger effect on your life. Do you prefer to use email and forums for support? Pick a host who focuses on that. Want someone on call 24/7? Pick a host who offers support in that fashion. Want to be able to install and update software with the click of a button? Look for a host who offers one click software installation and upgrades.

Basically, just identify what matters to you and realize that if you’re really going to use all of that disk space and bandwidth, you probably shouldn’t be using shared hosting.

There’s one other tricky question — Windows or Unix? I think this ones pretty easy though. Unless you plan on using ASP or ASP.NET (two web scripting languages), you should go with Unix. Many of you are now thinking to yourselves, “but I use Windows at home? Why wouldn’t I pick Windows?” The answer, quite simply, is that you don’t have to run the server so you just want to pick something that works well. And web hosts have been doing Unix hosting forever. It just works. Plus the cool SysAdmins all know it inside and out and can make it do crazy things.
So, we’ve figured out that you will:

1. Pick shared web hosting
2. Ignore disk space and bandwidth and go with the host that fits you best
3. Pick Unix (unless you need ASP or ASP.NET)

Basically, now you just need to shell out the dough and start building your site.

The next lesson?

Do NOT use Microsoft Frontpage.

It’s not that it’s a bad product. It does it’s job really well. It’s also a pain to support, has a startling tendency to decide to change your page without asking, and a tendency to become corrupt and break your site entirely. Plus, Microsoft is done with it. So it’s a dying product anyway.

Save yourself the time and money and grab one of the many nice free editors. Nvu is a good one.

Now just build your site. Try not to reinvent the wheel. If you picked a host who has one click installers, try something like Gallery2 for image galleries, or Wordpress for a blog. You don’t need to do it yourself — the one click installers can make it very easy to build a site without much work. However, make sure you stay up to date. If your web host offers upgrades or patches for your installed apps, take advantage of them when they become available.

If you don’t, you could login one day and see that your site says:

“Haha you suck you’ve been h4xx043d”

In other words, some ingenious little kid has hacked your site because you’re using a 4 year old version of some guestbook software your Uncle Dale (who smells like fish) gave you. You curse to yourself, “I’ll always upgrade from now on, now I just need to get my site back.”

Which, because you follow my last lesson, is a piece of cake.

Always back up your site on your hard drive.

I cannot tell you how many sob stories we hear about customers who are running their “business” off of their account, but don’t have their own backup. When their site gets hacked, they expect that their web host will have a copy of the file they uploaded 5 months ago. Here’s a little secret: most web host do have backups, but they are rotated out quickly (daily, weekly, or semiweekly). What that means is that if you get hacked while you’re on vacation, you may come back and find out that the only backup your host has is of the hacked file itself.

Thanks Uncle Dale, you fishy bastard.

Backup your files. Keep a copy on your hard drive. If you use a database-driven application, ask your host (or, better yet, look in their knowledgebase or forums) if there is a way for you to get backups of your database.
So, you want to build your own little site?

1. Pick shared web hosting
2. Ignore disk space and bandwidth and go with the host that fits you best
3. Pick Unix (unless you need ASP or ASP.NET)
4. Don’t use Frontpage
5. Backup, backup, backup

That’s it. Everything else you really don’t need to care about. And, by the time you need to care about it, you’ll be a superstar web expert and won’t be reading silly stuff like this.

In Part Three, we’ll disuss what to when you plan to make your living online.

So … you want to have a website? Pay attention. I think I can help you.

For a while now, I’ve been thinking about trying to put my small amount of web hosting knowledge to use in an informative and educational way. Whenever someone says to me “umm, what exactly is web hosting” and I try to explain, I realize that making the leap from the idea that you can have a website (which to most people, is just a domain name) to the actual idea of having some disk space and cpu cycles on some box out in the world isn’t an intuitive leap for people.

After putting some thought into it, I’m not too surprised. Web hosting isn’t obvious or intuitive. Particularly shared web hosting, where you’re actually splitting up resources with some number of other folks. Web hosting can be complex and confusing, with competing hosts simply trying to one up each other overselling their resources. In the end, for most folks, they look at one plan and say “Holy crap! 500 GB of disk space! 100 GB of bandwidth! Unlimited email addresses! That’s awesome!”

And two years later they’re still using 10 MB of disk space, 50 MB of bandwidth, and 2 email addresses.

The bottom line is that most people don’t need the maximum amenities. That’s what web hosts are expecting: the average customer is going to be attracted by out of this world options and use 1% of them. It makes the shared hosting world go round.

How does all of this tie back into helping you get a website? Well, to have a website you need some sort of web hosting. We’ll divide them into three groups: free, shared, and dedicated. Your first major decision will be deciding which type of hosting you need. To determine that you need to answer one simple question …

What do you want a website for?

Seems like a pretty obvious — and easy — question, right? Well, what do you want your site for? Do you just want a domain name so that you can have email address like myname@iamsoooooocool.com? Do you want a blog like all the cool kids? Do you want a full-fledged site that you might eventually turn into a small side business? Or do you want an eCommerce site that will be your main source of income?

There’s all sorts of gradients and mixes here, but you can probably fit your idea into one of those groups.

I want a coolass email address!

If you’re just looking for a cool email address, well, then you actually don’t even need a website! What you probably need is just an email plan, where you can get a domain name and have it hosted somewhere that will host email for you. These plans run as cheaply as just a few bucks a year (plus whatever you pay for your coolass domain name).

I want a blog!

If all you want is a blog, then you’re in luck! Like the email plan, you probably don’t even web hosting. Instead, you probalby want to check out something like Blogger, or Windows Live Spaces, or Wordpress.com and get one of their fun, free blogging accounts. In many cases, you can even point a domain name at it.

See, two situations where people often think they need some sort of exhaustive web hosting, where what they really need is to just take advantage of some of the free (or nearly free) resources out there.

I want my own small little corner of the internet!

I want to get my business online!

Both of these options do require some web hosting. And we’ll discuss them in Part Two.

I’m into about day 4 of using Google Reader for my RSS feed reading. So far, so good. I’m getting through feeds much more quickly; I’ve even added a few more feeds because I’ve found it faster getting through feeds in the Reader interface.

Only a couple of flaws at this point:

1. If you keep the Reader open in a tab or browser window, it can seemingly get out of sync. I’ve had a couple of times when the reading window thinks the feeds are read, but the left column doesn’t, and moving back and forth results in things getting weirdly out of sync. I’m guessing it’s just a web-performance issue (as it’s doing a ton of HTTP/XML work everytime you read a bunch of feeds), but that’s something you don’t run into when you’re using a desktop client.
2. I miss being able to highlight/color code certain feeds or search terms. That’d be a nice feature, as right now, feeds aren’t differentiated particularly well when you scroll down.

Otherwise, I haven’t found a reason to move back to a desktop client yet. I’m rather impressed.

My friend Greg has started posting again and running his movie on his website, Project Working Stiff. It’s a funny movie, more of a romantic comedy than you’d expect from the subject matter. It has a couple of literal laugh out loud moments, and some really great indie rock/pop music.

Go watch it. If you like it, post about it, or tell someone else to watch it.

Heck, I might setup a little spot on this here site for you to watch it (except it means I need to setup a Brightcove account, which I don’t have time to do right now … but stay tuned).

I’m taking a couple days off from work and heading up to the beach for the day with some folks. Should be a nice break from the ColdFusion-rich days I’ve been spending at work. We’re working on a project to take ColdFusion users from being spread out across our Windows servers and move them to their own Windows servers, where poorly written code can’t take down other, non-ColdFusion pages.

I’m not a big fan of ColdFusion. I can understand why people would use it if they’re not particularly skilled developers, but once you know enough to use ColdFusion well, it seems like you’d want to use ASP, Perl, Python, Ruby, PHP … something …. anything else. ColdFusion runs through Java, so it tends to be slow when you’re running it through IIS. Making things worse, we’ve discovered that ColdFusion’s default JDBC-ODBC bridge is pretty much crap. When you get 12 concurrent database queries, the ColdFusion ODBC service dies. But not gracefully, it gets stuck in a state where it can’t be stopped or started. The box has to be rebooted.

Why does this suck royally? First, because ColdFusion users (at least those using our service) tend to write really crappy code without closing their queries and sometimes running queries within queries, and they can hit that 12 concurrent database queries pretty quickly. But the bigger gotcha, the bigger kick in the junk, is that ColdFusion is installed as a wildcard script map in IIS. That is every single page request, ColdFusion or not, goes through ColdFusion for ColdFusion to decide whether or not it wants to handle it. So when ColdFusion dies, NO PAGES GET SERVED FROM THAT BOX AT ALL. It’s really quite annoying.

Yes, there are some things we could do to mitigate it. The logical one would be to remove the wildcard script map, but that actually breaks ColdFusion (some wonderful work you’ve done their, Macromedia/Adobe).

So, we’ve actually decided to segment ColdFusion users and use a native JDBC driver rather than the ColdFusion JDBC-ODBC bridge. I’ve spent the last few weeks of my life on this, moving customer sites, testing, writing Perl code to automate the process. It’s been fun taking existing ColdFusion DSNs and recreating them in the new format. It’s been more fun finding out the various little things that the new JDBC Microsoft Access driver doesn’t support that the traditional ODBC driver does. (Hey, for some ridiculous reason you’ve got an Access replicated table? Fantastic, the JDBC driver won’t read it. Hey, you’re using RND in your query? Fantastic, the JDBC driver doesn’t support it. Hey, you’re using a raw ‘Yes’ in your query to match a checkbox column. Fantastic, change your query to != 0 to pick up the positive values.)

It’s been a long few weeks, but it’s gone pretty smoothly, all things considered. But I hate ColdFusion, and I hate Access.

Thus, I’m off to the beach. Where there will be no ColdFusion. And no Access. Just my iPod, newly loaded up with all of the episodes of the Band in Boston podcast, as pointed out by Bostonist. I’ll be back soon feeling refreshed and ready to deal with more ColdFusion fun.

So I said I was going to eventually change my theme. I was fixing up some of the other sites I help run (Dewey’s House, Blair Wasdin Project, Chair Chuckers), and I got motivated and decided to start messing with my site.

I was browing the WordPress Community Theme Viewer and came across this one by Lasse Havelund and I was smitten. Not the perfect color scheme, and I need to make a new header image (gotta find out if The Gimp can read PSD files … I think it can!), but damn, it’s nice looking. I’ve been making some small tweaks here and there (changing where the comment text goes, some sidebar rearranging and changing, and the nifty digg! post icon). The big thing will be a new header image.

Of course, I just mucked around with Gimp for a while and it sucks dealing with PSD files (well, with layers), so I’m probably going to have to find someone who’s got Photoshop and actually make some real edits. Or maybe I’ll just go with my craptastic Photoshopping skills.

Thankfully, I know some folks who are good at this sort of thing. Because I suck at the Photoshopping.

A free beer for whomever can come up with a better header image.