Playing with Redis

Over the past few months, as I've been playing with different web technologies, I've dabbled with Redis. I really like the idea behind it: a super fast, all-in-memory, key-value store (disk basked so that you don't lose everything in a crash, can replicate, get backups, etc).

I think what I like most about it is that it is does one thing very very well. It doesn't try to replace your relational database. It just focuses on being a super fast way to store little buckets of data. Now, granted, you can do really creative things with those little buckets of data, but, by and large, Redis is just unbelievably fast.

Today I had a chance to play with Redis using some pseudo-real data. I wanted to benchmark/simulate how it would perform at holding keywords, and then the references (ids) to logs that contain those keywords. I'm imagining using this to keep track of support logs (hey, ftp has been mentioned in support contacts 15x today) or maybe to track server performance (tracking loads of servers or which services are running, etc). Tons of systems can do this. What Redis does really well is doing it fast (have I mentioned that?) and doing really awesome set geometry (intersections, unions).

For instance, imagine you wanted to keep track of support contacts. You look for keywords coming in via email messages (ftp, mail, outage, whatever). That's easy. But now what if you wanted to get the overlap where ftp was mentioned in the same email as cancel? That's harder. In MySQL, you might have to do some smart sub-selects, or joins to a bunch of mapping tables. It works fine when you've got a small number of rows, but the performance gets progressively worse. You probably also end up writing some ridiculous queries to make it happen in a way that doesn't end up going off to disk and killing your performance entirely.

With Redis, it's dead simple. I mean seriously simple.

First, you install Redis. It's about the easiest thing I've done. Untar, make, run. I didn't even bother to make any config changes.

Next, either via script or via the command line, you start adding data. For the support keyword example, you might do something like:

sadd ftp id_1
sadd ftp id_2
sadd ftp id_3
sadd ftp id_4
sadd cancel id_2
sadd cancel id_4

Basically, messages 1-4 mentioned FTP. Messages 2 and 4 also mentioned the word cancel.

To get that intersection, you follow that up with:

sinter ftp cancel

And that's it, it'll tell you id_2 and id_4 contain both words. Done and done.

Of course, that's a pretty simple example. I threw together a test script that generated 80 random keywords and 50000 random message ids for each keyword. The random ids were all in the same range (1 - 999999), so there's a good chance of some overlap, but not a ton. This seemed like a pretty good test of Redis.

It took, on average, about 4.5 seconds to insert 50k bits of data. So for the total of 4 million entries, it took about 6 minutes. That's pretty darn fast.

But what's so much more impressive is the intersection work. To get the intersection between two sets of data (which was usually between 1500-2500 entries) it took …

.05 seconds

Seriously.

And that, right there, is why Redis is f'ing awesome.

Eric Cantor, Douchebag →

"CANTOR: And what airlines have done is have stepped in and said, well, if we’re not going to pay that money to the federal government, we’re going to keep it towards our own bottom line. And I guess that’s what business does."

Oddly, Cantor seems to be unintentionally making the progressive argument about corporate taxes here. While conservatives generally argue that cutting business tax rates will lead to companies passing on savings to consumers and hiring more employees, progressives argue that corporations will just pocket much of these savings. The FAA shutdown offered an ideal test case for this question, and it seems that even Cantor agrees that pocketing tax savings is “what business does.”

(Via ThinkProgress.)

ThinkProgress pretty much nails it. Time and time again, the "market" has had the opportunity to police itself, to do all of the good things it says it does when unfettered.

And now we're all financing a huge bailout, a housing market that is gutted by a bunch of assholes betting on defaults, and a Congress that just passed a debt ceiling package on the backs of the handful of entitlement packages that have actually made a massive difference in the quality of life in the United States over the past 75 years.

The good news, though, is people are starting to get it:

A couple of weeks ago, congressional scholar Norm Ornstein dubbed this 112th Congress as the “Worst. Congress. Ever.” And after the combative and exhaustive debt debate, it appears the American public agrees with him. Per a brand-new New York Times/CBS poll, 82% now disapprove of Congress’ job, which is a record high in that poll. A CNN poll also had Congress’ disapproval at an all-time high of 84%.

Thank you, America, for showing some brains for the first time since 2008.

It gets better.

the Tea Party’s fav/unfav is now 20%-40%, compared with 26%-29% back in April; and by a margin of greater than 2-to-1, respondents said creating jobs should be a higher priority than spending cuts

Sanity.

Now, the bad news: President Obama will take this as a sign that he and his fellow Democrats need to show some more bipartisanship. So expect that, in the coming weeks, they'll compromise on a "jobs" bill that will not create any new jobs, but will reduce taxes on people making a lot of money and voting GOP. Because, you know, Compromise is better for American than Jobs.

Shocking: Nick Reruns from the 90s Do Big Ratings →

"According to Nielsen, the midnight-to-1 a.m. combo of All That and Kenan and Kel drew roughly 600,000 viewers (of all ages) Monday, compared to the 374,000 viewers who caught Lopez Tonight on TBS in the same hour. It also bettered the 559,000 viewers who caught an Awkward rerun on MTV at midnight, as well as the roughly 500,000 people who checked out reruns of The New Adventures of Old Christine and HIMYM on Lifetime"

(Via Vulture.)

This is in no way shocking. In fact, it's only shocking that it hadn't been done before this. As a child of the 80s (and a teen in the 90s), I was there when cable tv exploded. During that time, the smaller UHF network and cable networks flooded their time slots with reruns of 70s and 80s tv shows (and, later, 90s shows).

I seriously believe I've seen every episode of Happy Days ever created. A show that aired a majority of its episodes before I was 3. (Same with Laverne and Shirley, The Jeffersons, etc).

Nick at Night, the good old channels 56 and 38 (in Boston), old school USA Network. They aired every single semi-popular show from the 70s and 80s ad-naseum. Today, with reruns on related cable networks, Netflix and Hulu, DVD, it's much much easier for someone to keep up with their favorite tv shows.

So, as someone pretty much sitting in the every advertisers target demographic, I've seen every show from the 70s and 80s, and pretty much every modern show. And I can catch them whenever I want online or on-demand.

What's missing from this picture? The other shows that aired during the 80s and 90s. The shows that padded out networks like Nickelodeon and USA. The shows that have been incredibly hard to find and that would cause almost any 25-35 year old to stop what they were doing and sing along to the theme song.

When I heard TeenNick was airing these shows, I set my DVR to start recording Clarissa Explains it All and Doug. I still knew the theme songs. I remembered character names.

Apparently everyone else did, too.

Just wait until Hey Dude and Salute Your Shorts are on. Ratings gold.

Spotify: For all of your musical shame

I've been playing with Spotify for the better part of a week and I think I've decided that, at least for me, it's not an iTunes replacement. The music that I really like, I will still buy it via iTunes or Amazon. For one thing, it's really cheap (Amazon runs those $5 album deals all the time), and two, it means that I'll always have music available when I'm out of range of any signal (without paying $10/month). I also don't want my music tied to a company that could go out of business, or lose rights to certain artists, or whatever horrible licensing catastrophe the RIAA can come up with (c.f Netflix Streaming, it's lack of new titles, and ever rotating collection of long tail titles).

What Spotify (and, presumably, all of the other streaming music services) is really good at is listening to the music you don't like enough (i.e. you're too embarrassed) to keep in your iTunes library. When you're playing foosball and someone references Michael McDonald, you can go back and listen to "I Keep Forgettin'". When you hear some 90s R&B on the radio, you can play yourself some Jodeci and Joe Public. You can binge on Dan Folgelberg and the Little River Band.

Spotify fills in the gaps in your library with the music you would never really buy. Because you're too ashamed of it.

Which is awesome.

And also, likely, its downfall.

If Spotify (or Rhapsody or any of the other streaming services) really take off, why wouldn't Apple (or Amazon) get the same streaming licenses that Spotify has? If iTunes let you stream any music off of the music store (and create playlists, tag songs, etc), wouldn't that be about the best complement to buying music? Sure, there are some folks who will buy less music (trading it for whatever monthly fee Apple charges), but a whole bunch of people will stream a bunch of music, then buy more (when they realize they've listened to Hall & Oates 10 times in a row).

This seems like the logical next step for iTunes/iCloud (as well as for Amazon's MP3 Store/Cloud Player, and probably for Google Music, too). And a likely death (or, in a best case, acquisition) for the streaming music services. One of them will probably survive, just as an alternative to the Apple/Amazon cartel.

So enjoy it while it lasts. In a couple of years, it'll be part of iTunes (like everything else).

One Month Away ...

First real vacation in a couple of years.

Why Women's Soccer Might Make Soccer Work in America

If you are remotely sports-minded, the timing of this post is probably a bit too on the nose. The US Women's National Soccer Team just played what may have been one of the most exciting, frustrating, and ultimately American games of soccer ever played. However, these thoughts have been percolating around in my brain for a bit, and the game yesterday just crystallized that my premise is right.

For the average American, the Women's World Cup is a more interesting soccer tournament than the men's, and might be the event necessary to get your average American sports fan to watch soccer.

Image from the NY Times (I think)

With almost any other sport, Americans will not accept the women's version. Volleyball and gymnastics might be the exception, but those are not major sports and Americans will only care about them every four years.

But Women's soccer, particularly this team in this event, might help soccer catch on for the rank and file fans who generally spend their year moving from their local baseball team to football Sundays to the NBA and NHL playoffs. They might now spend a morning watching a Premier League game, or tune into a Men's national friendly.

• The World Cup is played at a high enough level that the average fan can see it's a skilled sport. This, alone, doesn't seem like anything particularly insightful. The World Cup, an international event that, like all good international events, plays on patriotism and jingoism to get you artificially behind a team you didn't know existed two days ago. When you turn on a Women's World Cup game, you can tell you're watching something important. The same thing goes for the Men's World Cup. Or the Olympics. Or your average EPL or [insert other league here]. Pretty much anything other than MLS. Americans like the best. The World Cup is the best.
• The players in the Women's World Cup are not as good as their counterparts in the Men's event.Now we are getting somewhere. While this doesn't sound like a positive, it is in this case. The Men's World Cup is full of supremely talented teams who don't play together enough. That leads to a very conservative game, a game that is spent almost entirely in the midfield, touch passes, reversing field, methodically moving the ball, probing for a weakness in the defense. Occasionally, you'll get a run by an offensive minded player, followed by a counterstrike by the defense, and then another 30 minutes of midfield play. It is this play, the generically boring, possession-based offenses, that bores your average American to death.Top leagues like the English Premier League, or really, any of the UEFA leagues, aren't like this. They have inventive and interesting offense, and a game that has a tremendous amount of back and forth. This is because these teams play together (basically) year round. They know each other, and their opponents, and they aren't in what amounts to a single elimination tournament. But, they're also foreign, with odd chants, and odd names, and few Americans.

  The Women's World Cup finds a niche the Men's doesn't. The players are great, but not so good that they have the ability to control the ball for minutes at a time. There are many more turnovers, fewer successful passes, and this leads to a significant amount of offense. Teams make runs all the time. The ball travels from one keeper to the other keeper in seconds, not in minutes. It is a less precise game, and, therefore, becomes far more interesting.

• The Women's National Team is good; The US Men's National Team isn't.And here's where our patriotism kicks in. Your average fan, when turning on a Men's National team game, particularly against any nation that isn't found in the Caribbean, can tell that the Men's team is in for a fight. Or, worse yet, in for a drubbing. They just aren't that good. Someday, that might be different, but it's not different today. Americans do not like cheering, or watching, a team lose. So, instead, we just don't watch.But the Women's team is one of the best in the world--or possibly, the best in the World. They won the Cup in 1999 (which almost everyone remembers), and have been consistently good. When you turn it on, you know you're watching a team that can win.

  (Why didn't soccer catch on in 1999 after the Women's team won the World Cup? Well, it did, a little. But, mostly, because it was 10+ years ago without 10+ sports networks, HD cable, and online video. You had to struggle to watch soccer in 1999. Today, you can turn on ESPN and have a broadcast of the Premier League that is done specifically for the American audience.)

The best teams in the world, playing an exciting (if sloppy) brand of soccer, in the biggest tournament in the world, and the American team has a shot at winning. This is why Women's soccer might turn Americans on to watching soccer on TV. Yesterday's 2-2 Penalty Kick victory over Brazil might have been the tipping point. It had everything Americans love, and rather than rehash that, I'll point you to American McCarver's recap of the game.

This tournament, particularly if the US team can beat France and move into the finals, might be enough to get fans to check out another soccer game. If ESPN is smart, they'll start showing recaps of Premier League games, pitching the upcoming EPL season, and pointing hungry soccer fans towards something besides MLS. Americans simply are never going to get behind MLS. We don't watch second rate leagues (at least not in large numbers). Hardcore soccer fans will watch MLS the way hardcore football fans watch the CFL or Arena league: they just like the sport and will do anything to watch it.

Casual fans, hooked by this Women's World Cup (and, in particular, this team), should be spoon fed EPL games, in hopes of growing a larger, American soccer audience. This time, it might actually work.

Google Dumps .co.cc From Their Results →

Google has removed over 11 million .co.cc websites from its search engine results pages on the basis that most of them are far too "spammy".

The .co.cc space is not an officially authorised second-level domain like .co.uk or .com.au. Rather, it's offered independently by a Korean company (http://co.cc/) that just happens to own the domain name .co.cc.

Huh, why would a company want to offer a fake registry? It's not like .co.cc is particularly attractive TLD.

The .co.cc "registry" offers single sub-domains for free, and enables customers to bulk-register 15,000 addresses at a time for a mere $1,000, or about seven cents a name.

Oh, I get it. It's so they can sell thousands and thousands of domains to spammers and phishers hoping to avoid the Google Ban Hammer (or to avoid Cloudmark or any other spam services).

Good for Google. It's pretty ballsy to dump 12 million websites/domains from your results. Now, just start dumping any site that puts up more than one AdSense block on a page, and we'll be in business.

(Via The Register.)

Home

We do 4th of July right in Rutland.

Irony. →

DreamHost Status » Blog Archive » Some FilesForever files lost: "Some FilesForever files lost

Unfortunately  we have some lost some files hosted within the file forever server. We have tried to recover as many files as possible however, some have been permanently lost. If you are still missing files on your account, please contact our support team by submitting a support ticket. We will provide refunds for any files that were paid for and lost."

I generally like DreamHost, but that's a little funny, right?

They may want to rename that service to "FilesForAWhile" or "FilesForeverInLittleKidTerms".

I thought this was pretty amusing too.

Is it any coincidence that just hours after Mensa went down, FilesForever ceased being aptly named?

Grantland: 2 Weeks of a Noble Endeavor

On June 8th, Bill Simmons launched his latest ESPN endeavor: Grantland, a sports and pop culture site. "Wow!", you say sarcastically, "isn't that exactly what Page 2 is?"

Yes, yes it is.

But Grantland is more than Page 2, for a few reasons:

• It is focused on long content, not just short attention grabbing blog posts
• It has attracted a really solid set of known writers (Simmons, Chuck Klosterman, Dave Eggers) and lesser-known writers (Bill Barnwell, Katie Baker)
• It's not plastered in ESPN's incessant branding and cross-promotion

At the core, Grantland seems to be an attempt to prove that if you generate good, sustainable content (even if it might be magazine article length) that the audience will come. And that makes Grantland a noble endeavor.

So, after two weeks, how does Grantland look? Let's start with the bad.

As a web property, its design is, well, I don't want to say excruiatingly bad, but it's pretty bad. The layout with new content appearing at the top is reasonably blog like, but without any of the markers that give you a clue about what's new since you last visited (or even just what's new today). The intermingled blog content (where sometimes it seems blog posts make the front page, but other times they don't) adds a bit of confusion.

Those are reasonably minor quibbles.

The site itself looks like it was designed in 1997. That's ok, I think, since they're going for the old-timey media feel (or at least I think they are). Except it ends up looking pretty ass-like when you end up with a giant Subway or Klondike ad in the middle of a page.

The use of footnotes (which is a Simmons favorite) is fine. The footnotes showing up in the right column, in line with the reference is a clever idea that sounds better than it works, especially if you use a service like Instapaper or ReadItLater (perfect for the longer content of Grantland). Footnotes are called footnotes for a reason -- the bottom of the page is *always* the bottom of the page.

Oh, and no full text RSS feeds. Seriously, it's 2011.

However, the the poor-to-middling site design can be completely overlooked if the content is as stellar as I think the team at Grantland wants it to be.

So far, sadly, the answer to that is that it is not uniformly great. But there have been some bright spots. Some truly, supremely, worth the experiment already bright spots.

Tom Bissel's incredibly thoughtful, "review as commentary on society" review of the video game L.A. Noire was the first article on the Grantland site that really met the high bar the Simmons' team is aspiring too. There are almost no mainstream outlets that would devote 5000 words to a review of a video game, unless it ended with the conclusion that they cause all of society's ills.

Charles Pierce's recollection of his time at The National is just the sort of well-written piece that doesn't really get written any more, or if it does, it's on some backwater blog that you hope you catch a link to on a Twitter. And it really was the perfect entree into what is probably Grantland's signature piece, to this point, the Tom Shales-ian oral history of The National.

The oral history piece does a few amazing things that I can't imagine flying on ESPN. It allows some ESPN folks to crap on other ESPN (and non-ESPN folk). It spends thousands of words reliving the days of a long departed sports daily. It makes it interesting.

If Grantland can launch one or two of these pieces a quarter (and, it's somewhat telling all 3 of these hit in the first couple of days), then it may not just be a vanity outlet for Bill Simmons, but instead a place where long-form content can go and actually be read.

If Grantland has more of the, let's say, spotty content that has filled its "pages" since the launch, I think it'll end up as just another Page 2. I'm hoping that the Chris-Jones-and-Wesley-Morris-like articles (two authors who I have enjoyed elsewhere), where we take a simple sports topic and try to turn them into something more poetic (or simply, purely, less readable), find their way to an editor who can reign them in.

Ironically, that editor may be Bill Simmons, and that hasn't proven to be one of his strong points.

I'm actually hopeful that each month Grantland will spit out a couple of "I'll read that once a year" articles mixed in with a few "did ya read that one yet" articles you share with your friends. Mix in a slightly better site design and I think it'll be a success.

By success, here, I mean something I'll go to and know that I'll be able to grab a good article or two to read on a plane or subway ride (or more likely, on the shitter). I'm not sure, beyond the Bill Simmons articles (which have lost some of their sheen when put next to better writers) that there's an audience for the site, but I'm really hoping I'm wrong. (I'd love to see what their webstats look like.)

There should be a place in this world for a site like Grantland.

← Older Newer →