The Self-Fulfilling Prophecy of Bad Release Processes

No Comments

If you need to wait any non-trivial amount of time between completing something and seeing how well it’s
performing, you’re not going to be working on that project by the time you get your answer. When you do get your
answer, you’re not only going to have to refresh your memory on what you had been working on, but you’re going
to have to do the same on whatever else you had started working on.[1]

I will admit, I’m currently a bit frustrated with our team’s development and release process. In the name of stability, we have given up speed, agility, and, honestly, stability.

Ironic, eh?

When you’re working on web stuff, having a long cycle between when something leaves your keyboard and when it is live on production servers being poked at by real, live customers is a bad idea for a whole bunch of reasons.

There’s always something different about production than your staging environment. It doesn’t matter how well prepare you, it just happens. Different security, some small schema change, a firewall rule. Something. The longer you wait between the last time you worked on your code and when it is in front of customers, the more likely something else will change, too. And the less likely you’ll remember how your stuff worked.

Somehow, handing over code to the QA team is considered to be a sanctified handoff. “My part is done. I’ve given the code to QA.” The fact that it doesn’t quite work, has significant issues, or blows up is irrelevant. The division of labor between developing and deploying makes it far too easy to pass the buck and make the brokenness someone else’s problem.

And, I think worst of all, you take your eye of the ball. If you’re engaged in your release, feel responsible for it, you can do smart things like push it live, make sure it’s working, take a look at some logs to see if your assumptions are right. Maybe you speculated on some language or colors, and it’s just not converting. Change it.

When you’ve decoupled your development and release processes, you don’t have that fine grained control, that immediate responsiveness. Instead, you need to get someone off of another project, have them dive back into the code, put it through another release process, and repeat it all over again. A change you could have made in five minutes and might have benefited your company or customers ends up in another week long QA process where, inevitably, some inexperienced QA person digs up some completely unrelated bug, and your watch your life just waste away …

Yes, long, rigid release cycles are necessary for software products that actually have to ship code, or might destroy data, or blow away finances, or do really bad things™

But if you’re working on the web, all they do is make things worse. I sincerely believe that. They don’t make things more stable (too many other things changing; too much context switching to remember why you had that odd bit of code), or make your releases smoother (too much passing of the buck; too many people involved; too many places where someone can say “stop the presses” because their nose is out of joint that day). They just make people feel better, and they let people cover their asses, so that the self-inflicted pain of choosing that release style can’t possibly be blamed on them. It’s a self-fulfilling prophecy. By making the process to allow people to cover their asses, you ensure that they have something to cover them about.


  1. Andrew Morrison from the above-linked post  ↩

Clever Apps: Airfoil

No Comments

Rogue Amoeba’s Airfoil is one of those apps that you don’t really need, but once you have it, you wonder how you ever lived without it.

The premise is reasonably simple: you’ve got audio on your desktop computer, you want to listen to it somewhere else. For example, you’re running Spotify and want to listen to it as you clean around the house. Unless you’re paying for Spotify Premium, there’s no way to get the music out of Spotify onto your stereo short of plugging the audio of your computer into your stereo.

With Airfoil, you gain a bunch of options. When you run Airfoil, you can send the audio to any Apple TV you’ve got. You can send the audio to any device running the free Airfoil Speakers application — another desktop, a laptop in another room, or even your iPhone.

Pretty awesome.

But, for me, the coolest thing with Airfoil happened the other day. I was watching some TV off of our iMac, and was trying to be polite and keep the sound low.

“Self,” I thought, “why not use Airfoil and broadcast the sound to your iPhone.”

Brilliant. Except, the sound was a couple of seconds behind the video. Which was incredibly annoying.

The smart folks at Rogue Amoeba have it covered, though. Turns out, the Airplay format inserts a 2 second delay. But, if you run the Airfoil Video Player, it will delay the video 2 seconds, so the sound and video are in sync. Including for web videos. So, I laid back and watched some TV off of Hulu, while listening to the sound as it played through my iPhone.

For $25, I continually find uses for Airfoil. It’s easily worth it (and it works for PCs too).

iTunes Match: Two Months Later

2 Comments

Back in December, I wrote up a bit about what using iTunes Match was like out of the gate.

There were some gotchas that were a bit vexing at the time:

  • Album Art Syncing
  • Smart Playlists With “Limits” Not Working on iOS devices
  • Play Counts Not Updating Reliably
  • No Genius Playlists on iOS devices

Let’s take these one one by one

Album Art Syncing

If you’re using multiple Macs (or, presumably, a Mac and a PC), this is now about as flawless as it gets. The syncing of songs from one computer to another seems to be nearly perfect. If I update album art on one machine, it now seems to be on the rest within a reasonable amount of time. There was a period of time where that wasn’t true, where random songs would be missing artwork when streaming them, but that’s just not the case now. If a song has artwork on one machine, it does on every other one. So, one check in the positive column!

However, that’s not the case for iOS devices. Now, for a lot of songs, your album artwork will be there. Particularly for songs you play a lot, your iOS device of choice (I’m going to go with iPhone) will download and cache the data. But, if you’re playing a playlist or have your Music app on shuffle, you’re going to find that a whole bunch of songs don’t have album art. The iPhone will go get the album art when you play the song (so the next time it will have album art), but even that seems flaky, as the length of time that the art gets cached seems inconsistent at best.

It does look like (and I should stress that this just seems different to me, it might not be a change) that the Music app will now try to cache artwork ahead of time. In other words, if you’ve got a playlist, it’ll go grab the artwork for the next 5 songs or so, so that they’ll be there. I think it does this, smartly, in an attempt to always be ahead of you, so you’ll never see the ugly no artwork icon.

But it just doesn’t work that well, for a couple of reasons. First, it’s just unreliable. I’ve seen enough times where I’ll just get a few songs into a playlist and all of a sudden I’ll hit a few songs with no artwork, then a batch with artwork, etc. It just seems to hit or miss.

More importantly, the caching seems to render the Music app nearly inoperable. I’m not sure if it’s cases where it doesn’t have a network connection, or if it has a weak one, or something else entirely, but when it starts downloading artwork for a larger playlist, you might not be able to use the UI for 15-30 seconds.

Very un-Apple like.

Given that, nightly, the iPhone uploads a backup of itself, is there any reason that, during the same window, it couldn’t download all of the artwork you need for your music? My library (about 8000 songs, all in the cloud, 90% with artwork) has about 500MB of artwork. Explain why the iPhone couldn’t grab that over wifi when it’s plugged in charging at night.

Baffling.

Not-So-Smart Playlists

The “limit” feature that works so well in iTunes on the desktop simply doesn’t work on iOS devices.

Well … that’s not actually true. Those options used to work, pre-Match. They just don’t work now. You’ll see people online bitching about playlists having too many songs, or inconsistent songs. This is why.

Your playlist that limits to your 25 most recent songs? It’ll probably just have every song you have in it.

Still not fixed. Presumably, this would only be fixed with an iOS update (it must be part of the Music app itself), but you’d think that Apple could set the cloud side of iTunes Match to simply sync over the actual contents of your “50 Best Songs” playlist, as a temporary work around.

No such luck.

There’s another smart playlist feature that still doesn’t work with iTunes Match on iOS devices, due to …

Play Counts Not Syncing

This is actually broader than play counts, it is really all meta data. And, like album art, it works perfectly on the desktop. I’ve now got this great system where I can crank through unrated music while I’m working, and when I get home, anything I’ve rated highly is already in my four- and five-star playlists. It’s really, really nice.

But it doesn’t work for iOS devices. At all. Or, almost at all.

Sometimes, the first track in a playlist that you listen to on an iOS device will update it’s play count. Not it’s last played date. Just the play count. And only sometimes. And only the first song. Bizarre.

This obviously contributes to the smart playlist problem. If you’ve got a “Radio” playlist like I do (where it’ll play songs that haven’t been played in a while), these just don’t work. Until I listen to those songs on a desktop, they will haunt me in the Radio playlist, showing up over and over again, even though I’ve heard them 50 times.

No Genius Playlists on iOS Devices

Nope, still don’t work.

So, where do we stand?

Well, if you’re a desktop user with multiple computers, iTunes Match is flawless. It really is. Stuff. Just. Works. It’s very Apple-like, which is something you couldn’t say 8 weeks ago.

Actually, there’s still one problem with the desktop experience …

The iTunes Match error messages suck. Flat out suck. When you hit a song that they can’t handle, you don’t get a good explanation why, and in the worst case, sometimes Match will sit there and churn for as long as you’ll let it. In my case, that was until my iMac crashed (because I hadn’t noticed it had been going for a day or so).

I do believe, though, that this is the exception. That most folks with your average iTunes setup are going to be just fine.

Back to our regularly scheduled summary …

So, as I was saying, if you’re a desktop user (a laptop and an iMac, or a couple of laptops, or a Mac and a PC), iTunes Match is everything it is supposed to be when you bought it.

If you’re an iOS user, there are still some problems, and I don’t expect these problems will be resolved until we get a new build of iOS (5.1?). They’re not necessarily load or cloud issues; they seem to be fundamental application issues that need to be resolved.

And I’m optimistic that these will get resolved. Some of them (the metadata syncing) just seem like bugs, not fundamentally unfixable issues. There’s nothing on the list that jumps out as challenging engineering (other than the scale).

Even with the stuff that works intermittently, or not at all, iTunes Match is still worth it, even just as a backup of your music, a way to play music on your Apple TV without having to keep a computer on, and a way to get higher quality versions of the crappy music you ripped in 1998.

Or maybe you don’t need a higher quality version of “… Baby One More Time”. To each their own. I suppose.

01BritneySpears BabyOneMoreTime1998

Goals for Twenty-Twelve

No Comments

After spewing out twelve posts in about 4 days to catch up on my Top 10 Songs of 2011, I thought a bit about what I’d like to get out of my site over the next year.

It’s been an interesting year for me, as my site actually went through a period where it got real traffic (that’s what you get when you write about a new Apple technology). My post on iTunes Match got the most traffic I’ve ever gotten, something like 1200 uniques to this point, which is a couple of orders of magnitude more than I normally do. 1200 uniques isn’t a lot, but it is when you’re used to only getting read by friends and family (hi guys!). I kind of enjoyed it, so I think one of my goals is to write a bit more about techie/Apple stuff this year, see if I can keep up that momentum.

I think it’d probably also be smart to write a bit more about music. With Spotify and MOG and Rdio, there’s sort of no excuse to not be able to at least try out music, so I’ll pass more of that along as I come across it.

Finally, I think it’s time for ryantoohil.com to get a makeover. That’s on the todo list for this year, and I’ll try to post bits and pieces about my makeover as I go.

Top 10 Songs of 2011: #1 Typhoon - The Honest Truth

1 Comment

I really do just love this song. The Oregon-based band sounds a bit like debut album-Fanfarlo or Arcade Fire if you saw them in a tiny venue. One voice, a bunch of instruments, some plaintive lyrics.

If that’s all this song was, it’d probably still make my top 10.

But that’s not all there is. It’s the little touches.

The quiet voices that start the song before the guitar kicks in.
The horns coming in and making the whole thing seem bigger.
How simple the song seems with the guitar and drum beat and horns all echoing the simple, repetitive lyrics.
How wrong you are to think the song is simple, as it opens up both vocally and sonically.
How the entire song is preamble to the last 90 seconds.
The first female backing vocal just tipping the bands hand.
The chorus.

The final minute of this song is one of the most wondrous minutes of music in the past few years. There’s no way you don’t hear this song and smile. And what more can you ask for from a little bit of music?

Amazon’s little 30 second preview doesn’t do this song justice, so here’s a live version from Letterman. You’ll get goosebumps.