September 24, 2011 at 3:53 pm

How Smule’s Product Alchelmist Made the MadPad Reality Remixer

smule madpad ipad music app nick kruge

MadPad creator Nick Kruge, as seen through his own app.

You might not recognize the name “Smule,” but you probably know its work.

The company is responsible for many of the most popular iPhone and iPad music apps ever: the I Am T-Pain autotuner-for-the-masses, the excellent Ocarina instrument, and the official Glee Karaoke app, to name a few. Each focuses on bringing fun, engaging music interaction to people who might not be software engineers or Julliard graduates, but still want to have some fun with music.

The approach seems to be working; just about everything the company touches seems to turn to gold. At the time of this writing, four of the top 15 paid iPad and iPhone music apps bear its name.

MadPad, Smule‘s most recent release, is a video sampler, in that it lets you trigger not only audio samples by tapping a square on a grid, but also a video of whatever made the sound. By rhythmically tapping the clips, you can  make music out of just about anything (examples). Even clips we wouldn’t think of as inherently musical — videos of a baby talking, for example — magically transform into block-rocking beats.

It’s a lot of fun, and though it wasn’t developed fully in-house by Smule — it’s their first foray into publishing apps from a third party — it maintains the same level of user-friendliness coupled with real music-making possibilities that marks all of the company’s work.

We sat down with MadPad creator and Smule “Product Alchemist” Nick Kruge to discuss how MadPad came to be, the current state of interactive music, and what’s in store for Smule, going forward.

(This interview has been edited for length and clarity.)

Andy Cush, First, I feel like a lot of MadPad users are unaware that it started out being developed by you, then published by Smule. Could you tell us a little bit about how you created it, how they approached you, and how that relationship evolved?

Nick Kruge, Smule: I was an intern at Smule last summer, and in addition to that I was a student of Ge Wang, our chief creative officer and co-founder, at the Center for Computer Research in Music and Acoustics [CCRMA] at Stanford [where Wang is an assistant professor]. I was getting influenced by a lot of the ‘Smule-ian’ ideas: social music, mobile music, and that sort of thing, and I really took to it. After getting the internship, I continued to learn a lot about how to make apps — how to really search for the heart of an app — and about what will really connect with people.

The following school year, my second year in the master’s program at CCRMA, we were working on a project for the Mobile Phone Orchestra – an iPhone-based performance group. The theme of this concert was audience participation, so we were going to make an app based on ‘Mad Libs‘ — Mad Libs for audio, where the audience would contribute sounds, and we would record them and stick them in an existing song blueprint.

When we tested it, we noticed that while it was fun, it wasn’t quite as engaging, because you couldn’t tell who made the sounds. You couldn’t look around and see. That’s when we started to develop the technology for MadPad as a video sampler. At that point it was just because we wanted people in the audience to be able to see who made the sounds that were contributing to the project. But once we got things going, we realized how fun it was to trigger sounds on the fly.

The [Mobile Phone Concert] happened, and it went as planned, but at that point I started to take MadPad on my own. I moved it over to iPad from iPhone, and started thinking about it like a sampler — very much like an MPC with video. That was definitely one of the big inspirations for it.

As it was getting to a more polished state, pretty close to where it is now, Smule approached me about releasing it as their first published app, and simultaneously offered me a job. I brought this mostly-completed project with me, but then we went and did a bunch of revisions and user testing on the design, and we got to use the power of the marketing machine that’s behind Smule. So really, it couldn’t have worked out better. It’s funny that you mention the MPC, because that was one of my first thoughts upon using it as well. That’s an instrument that gets used by recording professionals on stage. Do you see MadPad being used by people who would consider themselves professional musicians, or is it geared more towards the casual user?

Kruge: When I was first developing it, I did have more of a power user mentality, and I see the power of video sampling, so it’s possible that in the future we’ll have some features that will make it more of a professional tool. However, I’d say that the Smule-ian vision in general is ‘for the masses.’ One of our core values is for things to not be too complicated, so that everyone can use them, so we’re focusing on trying to make it a great thing for the everyday user. It’s been interesting finding that balance [between appealing to casual users and professionals], and I think we’re going to continue to work towards it with future updates.

jordan rudess madpad screenshotIt’s been really cool and rewarding as a developer seeing what people have done with MadPad so far. I would love to see, given more professional features, what professional musicians could do with it, but it’s already been really great seeing the community. We have about five thousand sets that have been shared, [many of them publicly], and we’re going to build out the ‘featured sets’ section over the next few months so you can find a lot of what other people are doing with it.

Being me, just being able to go through all of them, what everyone has done, has just been amazing. Seeing what people are doing all over the world — you get these [literal] glimpses of people’s lives, and I don’t know of other ways that this really happens. We’ve seen everything from very musical stuff and interesting, innovative uses of the platform to goofy things like people with their families and little kids in Japan. We had a really awesome steel drum from somewhere in Central America. It really runs the gamut. The smartphone is emerging as an ideal platform for making apps that are fun and visually interactive, but also allow you to create music in the same way that a more professionally-oriented instrument would. What makes mobile devices so conducive to that approach, and where do you see it going in the future?

Kruge: The ubiquity factor is huge — and also the velocity at which these devices have spread. Over five years, there are already some tens of millions of them out there. So there’s the ubiquity in that sense — but also in that your phone is always with you. I feel like these sorts of things just lend themselves to casual users; we’ve seen casual games take off, so it makes sense that casual media, whether with video interactivity like MadPad, or just music, will follow.

The other big trend is toward interactive versus passive forms of entertainment. It’s not just about listening, it’s not just about watching: You want to get in there and experience it a little bit for yourself, to personalize it, and these apps are allowing that.

Even something as basic as a karaoke app is a good starting point for interactive music. You’re taking a song that someone else has made, but making it your own. In that sense, I think even things like Rock Band follow that path of interacting with music that you know. And now, Björk’s new interactive album is on an even different level. I think we’ll start to see even more things in that vein — more interactive albums, and I hope so, because they’re pretty awesome. these low-entry-level apps for music creation and interactive media is a trend we’re seeing, and it will continue.

It’s something I’m always thinking about, and I do think that it’s the next wave for music. I hope so, because I would love to live in that world. Smule has brand recognition that many other app developers lack, and each of your apps has done very well, commercially. As a developer, what does it mean to have not only good products but also a name that some people recognize?

Kruge: That’s a great question for me. I was posed the question: ‘Do I want to release this on my own, and take all the revenue? Or do I want to go with a company with a trusted name, and, at the time, three of the top ten music apps on the charts?’

What does discoverability mean in the app store? To me, and probably to just about everyone, the answer is: It’s everything. I started programming just a few years ago, and before that I was a full-time producer of ‘regular’ music. It’s very odd, I find, how much [the app store] parallels the music industry. You have these amazing musicians, just like you have these amazing apps, and there’s just no way to find out about them.

Even though it’s a problem many companies and startups are constantly trying to solve, this discoverability problem — it has so quickly become such a vast sea of apps out there. It’s just impossible to get discovered, it seems, and that’s really unfortunate. So for me, that choice [between releasing MadPad independently and having it published by Smule] was a no-brainer. Does Smule’s recognizable name come simply from consistently producing neat, interesting apps?

Kruge: I think it’s a number of things. First, Smule got rooted really early on in the iTunes app store. We also have always had really amazing marketing and, of course, really high-quality apps. It’s certainly been a lot of hard work, but it’s also been some luck as well and good timing to get to [where we are] now. Is there anything up your collective sleeves that we’ll see in the months to come?

Kruge: We have a pretty power-packed end of the year. We’ve been kind of quiet for a while, but the roadmap definitely keeps us all up late, every night. What we have for the end of the year is going to be pretty great, both in terms of MadPad updates and promotions, as well as new apps and updates to our existing apps.

YouTube Preview Image
  • Lindak

    Great interview! I learned a lot from this conversation. Thanks!

  • nb23

    I’m a big fan of MadPad, and this interview is a nice clear snapshot of how it came to be. Thanks, Nick, for your work! Likewise, Andy, great interview.