In the world of expressive electronic music instruments, Lippold Haken and his Continuum Fingerboard hold a special place.

Back in the 1980s – when streamlined, minimal MIDI synths reigned – Haken was working on his Ph.D. in Electrical Engineering at the University of Illinois. And he was also inventing the Continuum Fingerboard – a unique alternative to keyboard controllers that has the ability to sense the position of your fingers in three dimensions.

Haken now teaches in the University of Illinois ECE department in Urbana-Champaign. He’s conducted research in real-time computer music systems, music notation and representation, music technology for the blind, and analysis/synthesis/processing of musical sounds.

And, for the last 20 years, he has been making the Continuum Fingerboard and refining it as an expressive electronic instrument.

In this interview, one in a series produced in collaboration with Darwin Grosse of the Art + Music + Technology podcast, Lippold Haken tells the story of the origin of the Continuum, and explains why he thinks expressive electronic instruments require more than just new types of MIDI controllers. You can listen to the audio version of the interview below or on the A+M+T site.

Darwin Grosse: The Continuum Fingerboard (right) is one of the coolest pieces of gear that I’ve ever laid my hands on.

As I’ve talked to other people who are exploring the world of expressive electronic instruments & controllers, everybody points back to the Haken Continuum, and to its inventor, Lippold Haken, as the person who set the standard for what this was to become.

Lippold, why don’t we start off with a quick overview for people who aren’t familiar with your work – an overview of what it is that you’ve developed, when the project started and where you’re at in its current configuration?

Lippold Haken: If you look at a Continuum Fingerboard, it looks like a red surface with black lines, in the same sort of pattern as black and white patterns on a piano, except that the black areas and white areas are all the same width. It’s really a continuous surface.

It doesn’t have keys, it’s a continuous surface and the black and white key pattern you see is really just a pattern for reference.

When you play it, you have to place your fingers accurately, much like you would on a violin, or on many other continuous pitch instruments. Then, you can play any pitch you like, you can play any tuning you like, and so forth.

The big challenge in an instrument like this, is to get precise pitch in the horizontal direction.

If you want to be able to do the thing where you play an interval, like a perfect fifth, or a perfect fourth, or a major third, or whatever it is, and you want to hear the beats melt away as you roll your fingers, or increase beats as you roll in the other direction, you need an extreme accuracy – a fractions of a cent. Much better than the thousandths of an inch for the center of the fingers.

Since the early 1980s, I’ve been working on this, not only on the sensing technology, but also on the synthesis technology. It’s fundamentally different, as you know. On MIDI keyboards, you just start and stop things.

A MIDI keyboard basically is a velocity sensitive bank of switches. You start and stop things and there’s no interaction with the fine structure of the sound – it’s all canned inside the synthesizer.

On the Continuum, my idea was that I want to really be able to control the fine structure of the sound.

I’ve been at this since I was a grad student in the 1980s, and by around 2000, I’d actually sold one or two of them, and yeah, so I’m still at it. I’m sitting right now in my basement workshop looking at one of these things, the one that I’m testing out right now.

Darwin Grosse: Nice. Just a couple of months ago maybe, there was a conference of fans of the Continuum, right?

Lippold Haken: Yeah, so this was the first ContinuuCon, ContinuuCon 2016. The will be one next year again.

It’s actually not organized by me.

I’m still a one man company, and I build these in my basement, with help from my father-in-law, but I make my living teaching at the universities. I just don’t have the time, or really the skills to organize such a thing

The Continuum community, including especially Sally Sparks out in Asheville, North Carolina who  organized that conference.

Darwin Grosse: I had heard from a few people that came out of that inspired, excited, and anxious to do more work. That is clearly one of those events where the meeting of the club, really helped the club to become stronger, right?

Lippold Haken: Oh yeah. It’s great for me too, just to see … As a one man company, a lot of the times when you talk to people, you’re talking when they’re having trouble, you’re not talking when things are going great.

Darwin Grosse: Right.

Lippold Haken: It’s just so fantastic to come together with people, and really hear them play.

It was amazing to hear all the different playing styles that people have developed, because unlike piano, where you have the Hanon technique, or for violin you’ve got the Suzuki method. Here, you don’t have the same learning guidebook that everybody uses. Everybody’s sort of finds their own way, and so it’s very interesting to see the big variety out there.

Darwin Grosse: You talk about yourself being a one man company, which sort of belies the psychic weight of the work you’ve done.

Everybody in the electronic music field seems to know about the Continuum Fingerboard. How many of these have you made? How many have shipped out?

Lippold Haken: People always ask that, and actually I only have an estimate because I know how many printed circuit boards I’ve ordered.

From that estimate it’s about 700 by now. That’s over many years. As I say, I’ve been selling them for the last 16 or 17 years, and of course worked on it long before that.

Darwin Grosse: Right.

Lippold Haken: It’s an enjoyable thing. Now, when I say I’m a one man company, well, I don’t have any employees or anything like that, but there are people that have committed enumerable hours to furthering the Continuum. Most notably, Edmund Eagan from Canada. I work with him, I talk to him.

Both of us have day jobs that actually support our families, but if we have time, we’ll work together for hours a day whenever possible. He’s put in an enormous amount of time, and an enormous amount of talent into the project.

Darwin Grosse: Now, the Continuum actually has a sound generating system in it, right? Is that correct?

Lippold Haken: Yes, it’s got a digital signal processor in it, or it can even have multiple signal processors in it if you want that.

It has it’s own signal processor in it, and a big reason for that is that, well, we live in a world where sounds and the synthesis engines are designed for keyboards, and for the fine structures to be automatically generated from beginning to end.

We wanted to develop (a synthesis engine) that’s really specific for this. In a synthesizer you’ll have envelopes, like ADSR envelopes or something, and when you play a note, you trigger those and they change the sound over time, but it’s done automatically.

On the Continuum….your finger’s an ADSR envelope. In fact, it’s much more than that.

You’re really, in a very fine way, controlling the sound. The issue with that, though, is that with an acoustic instrument, very complicated things happen. Say when you do a vibrato on a violin, as you roll your finger, you got the flabbier and the bonier part of the finger touching the string, you really get timbre changes, in addition to just pitch changes or amplitude changes.

Those complicated relationships between what your finger’s doing and what you hear, are extremely important. They’re what makes an acoustic instrument so interesting to listen to for many hours on end. In the same way, they’re what makes synthesizers … Well, listening to one patch many hours on end is kind of boring, you usually use many, many different patches in order to make them more interesting.

The fine structure control is a huge deal. That needs very accurate sensing and it also needs, really, a special synthesizer.You don’t want pressure just to change an amplitude, you have all sorts of timbral changes with it. Similarly, the front to back position, or the pitch, also make very complicated changes.

The EaganMatrix synthesizer is something that was a brain child of Edmund Eagan, that’s why I named it that. It’s a matrix-based modular synthesizer that is very sophisticated and does a great job at that.

Darwin Grosse: It’s interesting that you put that much effort into the sound generation system, because when I’ve talked to other people, particularly the people that are involved in the MPE-based systems, they are focusing on the surface, the playing surface, and wanting to take advantage of other peoples sound generator developments.

Lippold Haken: As far as MPE goes, it is useful too for the Continuum.

Often, especially if it’s not an important part, and you’re like … If you have a multi-part thing, there might be some tracks that you can record in something that wasn’t specifically designed for the Continuum, and there’s some good results. People have used Omnisphere and various other third-party synthesis effectively with a Continuum.

But, really, the very best sounds are specifically made for it. It’s just too different. If a sound engine is made for the idea of triggering things, and maybe then having pitch bend put on them, that’s very different then what the EaganMatrix is, which is, this very fine structure control.

Darwin Grosse: Right.

Lippold Haken: MPE is great, and it’s useful for the Continuum too, because there are sounds out there that are immediately usable, but I’d say the vast majority of Continuum users do actually use the internal sound engine primarily. The reason for that is that it’s just so different.

They also don’t get rid of their keyboards – it just isn’t a keyboard, it’s a different thing. So they have their keyboard, and they have their Continuum. They don’t try to replace the keyboard with a Continuum, and they also don’t try to, in general, except in some exceptional cases, they don’t use the same sounds for a keyboard as they would for the Continuum.

Darwin Grosse: One of the things I know from talking to a lot of different instrument designers and instrument builders, is that when they talk about newly developed instruments, two names keep on coming up.

One is the Chapman Stick, because it’s stood the test of time, even though it requires a completely different technique. It’s come to have a level of acceptance.

The other thing that people point to is the Continuum. Again, I think because it has been around a long time, because you sort of set the stage for where other people would go. Also, because it requires a different playing technique, and it has a different kind of articulation, it’s viewed as a successful modern instrument.

What do you think it is about the Continuum, versus the many other attempts to come up with, “The new keyboard,” that has struck that chord?

Lippold Haken: Well, with the Continuum, it’s an odd thing.

In some ways, it’s high priced, which is due to the expense of sensing technology I use. It’s very accurate, but it’s very expensive.

High price means that there’s a lot of buy-in, and people think about it for a long time before they get one. They also, when they buy it, they realize it’s much like if you buy an expensive violin. You don’t, because you bought an expensive violin, just expect to be a good violinist.

The Continuum users tend to realize that. They tend to realize, “Look, if I buy a Continuum, that’s nice, that’s step one, but it’s going to take work.”

The internal presets of the Continuum have a lot of depth to them. It’s not like a MIDI sound library, where you try this sound and see what it does, and then you try that sound. It actually takes a long time (to explore the possibilities of a Continuum sound).

Some of them take a few weeks, just to learn the techniques that work for that particular preset, and then other presets are very different to play.

Sometimes they differ by very fundamental things, like one is more percussive than the other, so it’s a finger tapping kind of sound, rather than a rubbing kind of sound, or whatever it may be. There’s more to it than that.

There are some sounds, like a feedback guitar, that are challenging to play, but very beautiful. Similarly, on the Continuum – the sound engine, the physical model – there might be setup to do a similar thing to build up energy.

It really takes a very fine control, like it would for feedback guitar, to get good sound out of it.

I think, fundamentally one of the things I’ve done different, is I have put quality first and not price first, which is unusual. Is it a good business decision? I don’t know, but after 30 years I still got a day job, so you can decide!

The other thing is, I also really wanted to have the same sort of joy to it that an acoustic instrument would have.

I go to the NAMM Show, and almost every piece of software, almost every instrument claims that you’ll do great new things with it, and there’s no learning curve. I didn’t take that route. That’s not important to me.

It’s not good as an engineer to make something needlessly difficult, but it’s also not necessary to de-skill musicians. I mean, I don’t want to make another instrument where the main purpose is to replace acoustic musicians with a single person who doesn’t have to learn.

Like on a synthesizer, right? You can press the flute button, or you can press the trumpet button, and they’re about the same to play. Well, if you’re an acoustic flutist, even a world class flutist, people would laugh if in two weeks you want to play a trumpet solo with a symphony if you’ve never played trumpet. Even if you spent the rest of your life practicing trumpet, you may never get to be world class at it.

If you do get there, you’ve ruined your lips and you can’t play flute anymore.

In any case, there’s a lot of buy-in in acoustic instruments, a lot of digging into the depth of individual timbre spaces that the different instruments provide. I’m hoping and trying to recreate such a thing.

Now is this a needlessly hard? I don’t know. I mean, I look at my neighbor who’s a gardener, she digs around in the garden all the time, and is dealing with vermin, and is dealing with all sorts of issues. I could give her plastic flowers that would look nice all year, and would be a lot easier, but that’s not what she wants, right? She really wants to do this growing a garden, even though it is difficult. That’s sort of the audience that the Continuum is geared towards more. It’s a much bigger commitment thing.

Peter Pringle Plays Satie on The Continuum:

Darwin Grosse: That makes a lot of sense, and it’s something that keeps on coming up as I talk to different people. This sense that some of the difficulty of controllers, some of the precision that’s necessary, and then some of the expressiveness that comes with the instrument, is the thing that’s going to allow individual playing styles, individual virtuosity, individual articulations, and things that sort of identify the player to exist in the world of electronic music in a way that it kind of doesn’t do right now.

Lippold Haken: Yeah, so, well, for instance the Continuum, I have a few people that are percussionists, that use it purely as a percussion instrument, and I’ve got other people that do other things.

You can really specialize, in some sense, talking about a controller being expressive, it’s kind of like talking about a bicycle being fast. I mean, in the end, it’s the rider that really has to do it. The bicycle might help.

If you buy a Continuum, you won’t be able to play expressively instantly. Some of the presets are easier than others. Some of them you’ll see YouTube videos, “Oh, I just got this,” and some people with a good ear that can carry a tune, can do it pretty quickly for some of the presets.

Overall, it’s something that will take a lot of commitment. I’d like to say too, there’s sort of this confusion I think out there about, well, all these, “Not a keyboard,” controllers are all sort of the same. There’s certain things I’d like to mention, if we have time, that are really quite unique about the Continuum, and I think really set it apart.

People immediately see, “Oh, it doesn’t have keys,” but there’s much more to it than that.

One of the things we’ve already talked about is EaganMatrix, is that synthesis being there.

Another thing we mentioned is the special synthesis techniques, and really making complex physical models, not just using samplers and that sort of thing, but making quite complex reactions, so your fingers ‘shape the sound’ like an acoustic instrument. Because, if you play the same notes several times, the same way, it’s not identical. It’s similar here.

There’s also extreme accuracy and pitch. Now, I come from a classical violin, viola background, so there that’s always a challenge for me, because I’m not a great musician. This idea that you can roll your fingers, this idea that on an instrument you can play exactly to major thirds and perfect fourths, perfect fifths and all the other intervals, as good string players, as good woodwind players, as all sorts of acoustic players do, and not have this thing of, “Oh well, we pick our pitches from a set of 12.”

I think those things really do matter.

Does that mean that sometimes you’ll have Continuum performances where there’s flubs? Sure. Is that a big disaster? I don’t know. I can tell you from being a violin player, for the first years of my playing, the only person that’d want to hear me is my mom. I mean, it was horrible to listen to me.

That’s part of it. That’s not necessarily a negative thing.

In fact, this whole idea of having every note have its own pitch, and really have its own pitch, as in the acoustic world of most non-keyboard instruments there, I think is an important thing that people don’t even listen to anymore. Recently, I’ve had a lot of interest from India and it’s because these ideas still matter a lot there.

Pallav Pandya at ContinuuCon 2016:

Lippold Haken: There are two other things that I’d really like to talk about.

Other than just the accuracy of measurement, there’s also the speed of transfer of information.

I get it from both sides. I get people that think MIDI isn’t fast enough. I sort of have to agree, as MIDI is almost fast enough anyway, in most cases. Normally, when people hear delays and stuff, they blame MIDI because everybody knows that MIDI is kind of slow, and it turns out it’s actually somewhere else. Delay is almost always somewhere else, and even the sensing of the instrument they’re using, or operating system overhead, or whatever it is.

The Continuum does though. If you’re really … Even with a single finger, if you’re moving it quickly, or if you’re making an attack of a note or something, it uses 100% of the MIDI bandwidth. It very carefully uses that bandwidth to send the most important information, basically so that the synthesizer on the other side can know what the finger’s x, y, z position is. Even the internal synthesizer in the Continuum is actually a MIDI-based thing. I send MIDI to myself. Well, it’s just a way to organize the software.

MIDI itself is not the issue there, and I’m concerned because there’s so much talk about MIDI bandwidth being too slow. At the same time, most soft synths have trouble keeping up with 100% of MIDI bandwidth.

If you connect a Continuum to software, well, it’s prioritizing sample generation over interpreting MIDI and it expects only to get a few bytes of information per note. I think MPE is … I’m happy with it among other things, because it is MIDI. It’s not trying to do a totally new communication standard, it’s just sort of standardizing how do you set up your MIDI, and I think that’s great.

The very final point, and I think this is extremely important, and one of the reasons why special design is important, is if you’re going to control the fine structure of a sound, the attack of it is very important, the beginning of it.

In MIDI, we have this key velocity, and with MPE the use strike velocity. The whole attack trajectory, the whole way your finger first touches the surface is reduced to a single seven-byte number, and I think that’s a big mistake.

The Continuum, in the first five to twenty milliseconds of your sound, gives you a detailed trajectory of what your finger is doing. I think psychoacoustically, musically, that’s extremely important, and that’s a lot of where the fine technique is that you can develop with a Continuum. It won’t happen elsewhere. It’s also reason why you need a special sound engine to make really good use of that information.

People don’t realize, I think, that haven’t grown up playing acoustic instruments, how much control you have in your fingers, very light and very fine finger control over sound. That, I think a synthesizer generation just isn’t very familiar with.

Darwin Grosse: I appreciate you talking in detail about MIDI, the myth of MIDI being too slow, because you’re right, that a lot of times it’s something else that’s really causing the problems. Also, the idea of the organization of the data that comes through.

It’s interesting, because people that are producing MPE keyboards now are saying, “Hey, we have more data.” What you’re saying is that, especially because the expectation with a Continuum is that you’re going to be controlling the synthesizer parameters manually, that the articulation, and the organization of that MIDI data becomes even more important in that kind of environment.

Lippold Haken: Right. In the end the real job that MIDI has for the Continuum is, give the x, y, z, the pitch, the front to back position, the pressure of each finger as accurately as possible to the synthesizer, and as quickly as possible. That does take some thinking about it, because during some parts of the sound, you have an enormous precision. If you look at acoustic instruments too, it’s because notes last a long time and you can control things very fine detailed ways, you need 14-bit control. For instance, in the attack phase of a sound, actually you don’t need the 14-bit control, 7-bits is plenty, but you need it fast, you need very good time accuracy.

There’s all these different issues about exactly how do you encode things, what’s the most important thing to encode at any one time. I think the synthesizer manufacturer has to think about, but there’s a big problem with making MIDI super fast and allowing, whatever, an arbitrarily high data rate, because … The receiving end has to deal with all that data.

MIDI is an interesting thing, which in some ways, is a positive thing. It forces the instrument maker to be very careful about what you encode and how you encode it, if you want to give a detailed trajectory of what the finger’s doing.

I think that, in itself, MPE doesn’t directly address this. MPE addresses the whole issue of all this configuration mess. But there’s still a separate issue, for every manufacturer, exactly what do they want to encode.

I think one of the big things that I see happening now, is that people use the equivalent of a MIDI key velocity, they use a strike velocity, a single value that triggers envelopes, and automates much of the sound, and really takes away the major part of what it would mean to finally control a sound and really have expression that can be finely controlled. I’m very happy with MPE. I do think though, that there’s much more to the world than just encoding and I think they’re just making a device that can produce MPE. The device itself has to be very carefully thought out.

Darwin Grosse: Sure. Now, one of the things I like to do in interviews is find out how people come to become the artists, and makers, and builders that they are.

Your passion towards things like this high level of detail, your background in engineering, but also a clear understanding of musical terms, makes me really curious what is your background? How did you get started in both the engineering side, and the musical side, and how did those two things come to mesh together?

Lippold Haken: Well, my mom made me practice violin ever since I was a little kid.

We had a family of six and all of us had to get up early in the morning and practice. As much as I tried to avoid it, I would cheat and claim I practiced, and there was so much racket in the house with all the other kids playing violin, it was hard for her to tell. She eventually got us reel-to-reel tape recorders to record over, and she could spot check.

Then of course, I figured out I could copy tapes. Anyway, it was an ongoing battle, but in the end most weeks I practiced at least an hour a day for most of my growing up.

I never became a very good musician, but I learned a lot about it. I ended up playing a lot of orchestra, pit orchestras and what have you, and was exposed to a lot of different things. I had much more musical siblings that could be music professors and play in professional symphonies and such.

I ended up doing the engineering side of things. I knew a lot about music, I knew what my shortcomings were, I could tell a lot of these things, but I also knew what was possible, and I could tell the difference between what can I do, and what an expert do on this instrument.

By going into the engineering side of things, at that time, this was before MIDI, so we made some of the first keyboards they output ASCII code, which is an 8-bit code not too different from MIDI. MIDI was better thought out, I would say. Yeah, we did something like MIDI before MIDI, and built keyboards and such, because we wanted to try to make possible to hook any keyboard up to any synthesizer.

We built synthesizers when I was in college, but in the end, even with aftertouch, which was much more common back then, and these other ideas, there still was a lack of the most fundamental expression thing.

In violin playing, the most important expressive thing you have is vibrato, and that was missing. I said, “Okay, I want to build an instrument where vibrato is possible.” That’s where it started.

Luckily, due to my background, I knew that the whole thing isn’t worth it. If I can’t roll my finger on a surface and make beats melt away, it’s not accurate enough.

It actually took me a long time. It turns out it’s hard to make something that feels good under your fingers, and is super precise, and is polyphonic. It’s no small thing, so it took me a long time to get it figured out.

That’s where it started, and it was really thanks to my mom. I’m so thankful that she forced me to play and did that daily battle with me to get me to practice my violin, because I learned enough about it then, that I was able to build an instrument.

I used to actually practice Continuum quite a bit and play for people. I finally got good enough that the worst thing possible happened, people would walk up and I would play for them, and then they would say, “Oh, that’s what it does,” and walk away.

To be fair, great violin builders and such, I’ve known a couple now, they can all perform, but they only do it for themselves to test out the instruments and they always get the best performer possible to play in public.

Darwin Grosse: Now, your name has come up in a lot of different areas. When I did an interview with Carla Scaletti (Kyma), she mentioned that you were involved in some of the computer music synthesis stuff around which she grew up in.

Lippold Haken: Yeah, so both Carla Scaletti and Kurt Hebel (Kyma) are just brilliant.

What happened there is, early on I was building synthesizers, I met Kurt first, so I had the sound engine, but we needed a control computer.

At the end, at that time, the Z80 was considered a fantastic computer, it was an 8-bit thing. In any case, to control the synthesizer we built … Then, Kurt and I kept going at it.

Eventually I got to the point where I wanted to build a Continuum, I didn’t have the sensing technology and that sort of stuff figured out, but I also knew I really wanted a new synthesis technique. At that time, I thought that sound morphing was the end all and be all, and so I built hardware, or designed hardware in the early ’80s, that would be able to do sound morphing, and added a sine wave sound morphing.

Making a thousand sine waves isn’t the hard part. It’s figuring out how you control the amplitudes and frequencies so it sounds interesting, sounds high quality.

In any case, it turned out that Kurt had a lot to do with that. He helped in making the compiler for it, make all sorts of things for it, helped debug it, did lots of stuff with it. By then also, Carla came around and they figured out … Because Carla had this dream of general purpose music language, the Kyma system.

Well Kurt figured out, it’s just a few additions here, Lippold’s thing could implement your dream. That’s how that came to be. I originally had made it for a much more specialized idea of, well, I wanted to use a synthesis technique that needed a lot of different kinds of computations. That was expanded with Carla’s dream, and Kurt’s engineering knowledge to be the first platform for Kyma.

Darwin Grosse: Now, in the development of the Continuum surface, I assume that there is probably a long trail of tears getting to the flexibility and the accuracy, but not over accuracy. I’m sure that there’s a junk pile full of failed attempts, right?

Lippold Haken: I use this sensing technology, it’s Hall effect sensors.

I have tried many, many different tracking techniques, including capacitives and conductives – all sorts of different kinds of sensing techniques, and none of them could get the accuracy that I wanted.

I wanted to really to be able to tell the center gravity of a finger – much better than a thousandths of an inch, just about ten thousandths, or at least a five thousandths.

I kept coming back to these Hall effect sensors, the magnetic sensing, proximity sensing is super accurate. It was also one of the more expensive ways, because I had tried all these resistive methods and stuff, I just couldn’t get the accuracy out of them. In this, the magnetic sensing, these Hall effect sensors are good to many, many bits, and they tell you a distance.

Basically, in the Continuum surface, on the top you have this layer of nylon that’s manufactured into the neoprene. Underneath that is aluminum rods that are suspended on piano wire springs, and they have magnets crimped on the end. Then, the actual sensing is, when you press down on the neoprene, the rods you press, and the magnets move farther and closer to sensors.

Anyway, that whole setup is a very expensive setup. It has some nice things about it. The fact that it’s mechanical and it doesn’t feel like just a rubber surface…it’s nice.

Again, like you say, it took a long time. Even once I figured out, I’d sort of resigned myself, I have to use this very expensive technology for sensing, the other technologies aren’t accurate enough. Even after resigning myself to that, there was still this issue of, well now, we had to use rods and I could cover them with different materials, but actually the piano wire springs are great.

F=kx – the physics law is very accurate there. They were great for sensing, but for when you press on something, really, the totally linear pressure force is not ideal. It did take me quite a few years until Ed or I thought of using neoprene.

It’s kind of funny, because here in the basement I’ve got my old diving suits and they’re always there, I just never thought of cutting one up and trying it. In any case, so I took a long time till I did neoprene. The nice thing about the neoprene is it’s super uniform, you sort of have this princess and pea problem, that your fingers are so sensitive that if there’s any non-uniformity in the surface, you’ll feel that right away and it will be irritating.

The neoprene is super uniform. At the same time, it also is spongy and so if you … As you press harder it compresses more, unlike springs, it’s a nonlinear thing.

All your pressure gets transferred to the springs, because of the way this mechanism is set up, so you can still measure everything super accurately, but the feel of neoprene together with the springs, is just a very good feel.

Anyway, yes you’re right, it took a long time. Then there’s all these little things, even with having this particular setup, this neoprene is stretched over all these rods and there’s hundreds of springs in there and stuff. Well, if you want to service one of these things, or if you want to exchange a surface, or if you … Whatever it is. There was a lot of years of work on trying to make little improvements just to make a whole thing more durable and more practical.

Darwin Grosse: Well, I have to say that having tried it, it’s remarkable to hear what the mechanism behind it is, because when I played it, it seemed like somehow magic was happening.

Because you’re right, my expectation was that I would feel irregularities in the surface or whatever, and they’re just not there. The result is that, it’s sort of like takes this fear factor away, and you just get sort of … you pay more attention to the spacial awareness of your hands and stuff.

It’s really effective, as is that nylon covering, which gives it just the right amount of smoothness to be able to move around. It’s a really interesting way to pull that off.

Lippold Haken: Looking back, it’s always amazing. It took me a long time to figure out that these aluminum rods that are under these piano wire springs, that they … Until I had bought myself a microscope and started really inspecting them, I just didn’t realize how precisely they had to be machined.

Now I have a place that machines these things, they also make Swiss watch parts. I mean, they make very, very accurate machining, and that’s a lot of the smoothness in these mechanical systems. It’s just expensive, because every little part of it really has to be perfect. I always tell people, well compared to an upright piano action, the Continuum’s actually very simple. An upright piano, every key is different, all these things … Then, upright pianos are made in specialized factories, and that’s why they can do those cost effectively.

In the Continuum, we aren’t to that point yet, so right now it’s expensive to make, just because of all the machining and such. Every little thing, like the neoprene, just using neoprene was a big breakthrough, but then what exactly should its thickness be. Then there’s many different options in what kind of nylon to put on top.

Very fundamental things, like is it actually good to feel any difference in the blacks and white pattern, or black and red actually it is, because white just takes smudges too easily, it starts looking dirty. The black and red pattern, should there be any feel difference between black key areas and red key areas. Well, for some people they’re not even playing pitched music anyway, and they’re making sound effects, or doing some other thing with the Continuum. In that case, they either don’t care, or completely smooth is nice. There’s sort of this stage of Continuum playing where it’s nice to have that different textures under your finger.

Then the most advanced Continuum players tell me, “No, they really like it uniform,” because you don’t want to have this thing that depending on what pitch you’re playing, whether it’s a trill, or whatever little motion you’re doing feels different. You really want it to be uniform, you don’t want to have a preference for this key or that key. Especially because a lot of the times you’re actually playing in some other tuning, it’s not an equal tuning, so the touch feedback just gets in the way.

In the end, I think most of this is listening, and you just have to get the good ear. I have a whole pile here I’m looking at of experiments with just different variations of the nylon. All the neoprene’s the same in this pile, but there must be 40 or 50 different variations of what nylon type and how it’s marked that I tried. It’s been a long thing, but most of its been incremental improvements.

This is more like an acoustic instrument builder who’s always trying to improve their craft, rather than something that’s massed produced, where you a make major release, make a whole … They’re all the same, the next time try to fix any problems that you had in the previous release. This is all hundreds and hundreds of changes over many years of gradual improvement.

Darwin Grosse: That’s an important point, and it’s another thing that I’m always curious about when I talk to instrument builders.

I mean, especially something that requires a new approach to an instrument, right? I’ve seen some incredible players use the Continuum. One of my favorite was Richard Lainhart, he was like a hero to me, and the way he was able to do this beautiful expressive playing with the Continuum in conjunction with a Buchla, was just amazing.

One of the things that I’m always curious about for instrument builders is how do you know when you have an instrument? I mean, you’re going through all these experiments, it must be very difficult to say, “Okay, it’s an instrument enough that I’m willing to put it in the hands of a musician.”

Lippold Haken: It’s hard, hard to know.

You have to be careful as an engineer, or as an instrument builder, not to be too conceited. I mean, if you want to build an instrument that’s worth practicing eight hours a day, like say violin is or something, it better be pretty good, to expect that anybody would want to do that.

There’s really a lot of problems to it. The Continuum certainly became a real instrument, I would say, really got to … At the level that it is now. I dreamt of it, but I couldn’t do it myself. I mean, Ed Eagan is such an incredible performer, and has such a good ear, and is, between the two of us, we have all this technical knowledge of … Well, enough to build up the EaganMatrix.

He designed the whole thing and did the user interface. I did the low level and the DSP code for it. Between the two of us, we were able to do it. By myself, I could not have made it at this level. I would always be aware, just like I’m aware I’m not that good a violist or violinist.

In a similar way, I would be aware that it’s just not there. At this point, there’s a lot of things that were hard.

One of the big things is, say you’d invented the trombone, your first players are going to come back to you and say, “Look, even my gear shift on my bicycle will ratchet into the different gears. It’s easy to know you’re in the right gear. Just make the trombone slide also click into pitches, so that it’s not so hard to find the pitches.”

Seems like something like a great idea to a beginner, but no real trombone player would ever want that, not a skilled one.

The same thing happens here, like a lot of the times it’s really hard to tell. You can make a change, and it takes quite a while until you find out is this a good idea or not?

I’m not afraid of having it be difficult, but needlessly difficult is bad. Does this really make it something that gives more options to a very skilled performer, or is this something that just gets in the way, or what? There’s been a lot of things where, it just takes years of experience. We try something and then Ed plays it for quite a while, and sometimes he’ll come back and say, “No, forget it,” or other times, “This is really good.”

We’ve been down many, many different avenues like that. In general, an engineer would like to think, “Oh well, I can sense to this accuracy, and I can do that, it fits the bill, sell it, and make a million of them.” I really don’t think that’s a very effective way to make an instrument that’s on the level of acoustic instruments.

Lippold Haken on Continuum History at ContinuuCon:

Darwin Grosse: Since Continuum number one, what is the thing in the instrument that has changed the most over the years?

Lippold Haken: Well, the sound engine is extremely important.

Early on, we were using the Kyma system, which is very versatile and still is a fantastic system to use for real-time control. Again, it’s one of those systems that a lot of people shy away from, because they’re afraid of the learning curve.

I say, “Wow, if you want all the possibilities – learn the system.”

It’s very good at doing very fast real-time computations in response to what’s going on the Continuum surface. We were using that.

From using Kyma, we learned, or Ed learned, I should give the credit where it’s due, learned what are the most important mappings, the most important ideas of what your fingers doing and how it should effect sound parameters.

So, we thought, “How can we strip that down and make a system that’s much smaller than Kyma, make something that isn’t a general purpose language like Kyma is? Maybe it can’t do everything that Kyma can, but…packed in a convenient way for experimentation, specifically for the Continuum Fingerboard, and that would work on this DSP that’s built into it.”

That process was quite long. We started on that in 2006, and by 2008 I think all the Continuum’s had this built into them. That’s been a very important change.

Another important change has just been all the stupid little changes with the surface. I mean, you hit the nail on the head, just exactly how things are put together, exactly how things are machined, makes a big difference.

If you look at violins, if you look at a $300 violin made in China, one of the better ones, and a $50,000 violin, the only difference between those is in the details. In fact, a $50,000 violin probably looks old and kind of beat up, and the other one looks shiny.

In the end, a very good violin player could pick up the 300 dollar instrument and play quite good music on it, and really only a very experienced listener would even be able to tell, “Hey, this instrument is really limiting what they’re doing.”

It’s funny, because all these tiny details, between the 300 dollar instrument and the 50,000 dollar instrument, are different.

I’m trying to work on those details, and we continue to work on trying to get the details right.

Because, for the very best musicians and performers, it’s all those details that really, really matter, and make the difference between an instrument that’s really world class, and an instrument that’s just pretty good.

Darwin Grosse: Sure, sure, that makes a lot of sense.

Lippold Haken: One of the nice things is we’ve got a much higher quality Hall effect sensors that came out some number of years ago, and those have been great. It just does practical things, it makes it easier to play quiet. As most people know, quiet trumpet playing is gorgeous, but it’s actually quite difficult, it’s easier to play loud.

The same thing happened on the Continuum, it was just difficult to play quiet and I figured, well, in acoustic instruments that happens too. In any case, now it’s much … With these light-action Continuum’s that we’ve been making for the last years, it does make it a lot easier to play quiet, it makes it a lot easier to get the big dynamic range.

The biggest challenge now is that people are used to playing either acoustic pianos, or other instruments that take much more pressure, and they have to learn to use much, much lighter finger movements on the Continuum than you would on these other instruments, and really get the dynamic range out of it.

Darwin Grosse: You’re obviously very passionate about the sound engine that the EaganMatrix provides, but the Continuum has MIDI output built in, correct?

Lippold Haken: That’s correct, yes.

Darwin Grosse: And you also make a voltage converter for people who want to use CV based systems, right?

Lippold Haken: Yes.

I ran into this thing where people spent so much time trying to set up their CV right, and often didn’t quite make it.

At that point, it was just easier for me, or practical for me, to provide a CV that’s only for the Continuum. The voltage converter I make, really is just for the Continuum.

There’s lots of good CV’s out there, lots and lots of manufacturers make good ones. Use those for your keyboard. But for the Continuum, use this one. You can use other ones if you want, but use this one if you want to avoid the hassles of getting it set up exactly right. Again, with something like MPE, a lot of these problems would be made much lighter.

Darwin Grosse: Sure. For you though, I mean, even the voltage converter it has x, y, z, as well as, gating for four voices, but does that seem really limiting in comparison to what you can do with the EaganMatrix?

Do you feel like the EaganMatrix is really the voice by which you can express things with the Continuum?

Lippold Haken: In the end, the EaganMatrix takes that gate x, y, z and does amazing things with it. In that sense, they both sort of have the same initial input. All you’re given is the x, y, z of what your finger is doing.

The great thing about many analog synths – analog modulars – is that you can really get inside the sound. Especially before we had the EaganMatrix, we’ve had the voltage converter longer, it was always a big problem. Either people had Kyma, which was also very challenging, and then the number of people that are both technically skilled enough to use Kyma and also performance skilled enough to really take advantage of a controller, it’s a small number of people.

The voltage converter was nice, because it allowed this x, y, z information to get out into the analog modular world. A world that wasn’t totally sold on the idea that you just start, stop things, and it was also a world where you can re-patch your modulars so that the core of your synthesis algorithm, you really have access to it there.

As that’s a problem on many digital synths, you really can’t get into the guts of it. On a modular synth, if you want to use that x to control ten different parameters in different proportions of some complicated patch, you can do that. Where, in most digital synths, you can map a few things to something, but that’s about it.

This very complicated mapping that you need, often a nonlinear mapping that takes combinations of pitch, and pressure, and front to back position to map onto all the different parameters you have in your sound, or all the different filters and modulars you have in your modular, that’s something you can do in an analog world. It’s a lot of work, but it’s something that’s possible, and something that people are very familiar with.

For many years, until we had the EaganMatrix built in, I’d say the majority of people that are really successful with the Continuum, either were good at Kyma, or they used analog modulars.

The digital world just didn’t have the kind of control you would want. It’s not that MIDI isn’t fast enough, it’s not any of that, it’s more that … Well, as anybody knows, if you take a recording of somebody playing an acoustic piano quietly, and you turn up the volume on the recording, it doesn’t sound like a loud piano, it sounds like, well, like you turned up the volume on a quiet recording.

There are all sorts of things that change with amplitude changes, they’re not just brightness, not just what have you, they’re many, many different parameters.

In an analog world, at least you can get closer to that. That’s something that the modulars have always been good at. There are of course, many other challenges in modulars. They’re big, it’s hard to recreate the patch that you had going yesterday exactly the same, all these other issues. I mean, there’s many challenges, but there’s also many very good things about those kinds of systems.

Darwin Grosse: What do you see as the future?

I mean, we’re seeing now the movement with MPE keyboards, it’s bringing more expression into the general synthesis world, or electronic music world. We’re also seeing more people experiment and try out different instruments, or bring different instruments into their web of use.

I’m curious, what do you think represents the future for instrument interface in electronic music? I mean, you’re literally developing it to a certain extent as we speak, so I’m curious what is the direction that you see things going?

Lippold Haken: Well, I think there’s two mistakes that I hear a lot.

One of them is, keyboards are great, and keyboards … It’s just very well developed in the synthesizer world, the whole thing that you’re only starting and stopping things when you hit the key, and how hard you hit it is all that matters, frees you from all sorts of other considerations, and actually allows many kinds of music that aren’t possible with, say, with something like the Continuum, that insists on tracking every little thing you do.

Keyboards are great, and they’re always going to be in the future. Just like in acoustic instruments, keyboards are great there too. There’s a very big acoustic world, a very large variety of acoustic instruments out there.

In the electronic world, for decades anyway, it seemed to be a diminishing thing. That a generic sort of keyboard could control any synthesizer, and also this whole idea that you don’t have to learn new things. I think for the Continuum, the fact that when you play those different patches, that you really have to practice that particular patch, and particular articulations, and particular playing styles it was meant for is a big deal.

In electronic music, I do hope that people will get an ear for that again. I think what’s happened, is we have a generation now of performers, many of whom, never played an acoustic instrument. I don’t know a nicer way to say it than they don’t know what they’re missing.

Often I think what happens, people try one of these things that claims to be an expressive instrument, and in the end, it’s a weird term, right? How can an instrument be expressive? I mean, it’s a performer that needs to be expressive.

How can you be expressive without lots of practice? I mean, to me, that’s almost an oxymoron, like what are you expressing if there’s not a lot of skills involved?

What I’m hoping is in the future, is … Well, a continuation of this fantastic heritage of keyboards, but also more of this interest in really controlling the fine structure of a sound in the same sort of world class way that people control their voice when they’re singing, or when they’re playing acoustic instruments. I think it may happen, I don’t know, but I’ll do my part to try to keep it alive.

Darwin Grosse: Well, that sounds really exciting Lippold, I look forward to seeing the things that you come up with. I’m an avid fan of your work, and I appreciate everything that you’ve done to sort of move all of this kind of work forward. I think it’s fantastic, and I think it’s necessary for the world of electronic music, so I thank you for that.

Lippold Haken: I do have one other thing I want to say – we have a release of our firmware coming out. Since we both have day jobs, I hope it’s soon.

In this next release, one of the things I think that will be exciting to see for people is, we have a new support for ways to play extremely quickly. One of the problems is on keyboards, you just have to hit the right lever, which means that you can play very fast passages beyond what you could do on many other instruments.

Well, there’s been technique that was developed on the Trautonium by Oskar Sala for playing very fast, even a continuous pitch instrument and we hope to introduce those in this next release. Watch for that, and if you’re into playing fast, and still want to have perfect intervals and all that, it’s all very possible. I just want to give people a head’s up.

Darwin Grosse: That sounds great. Well thank you so much for your time, I really appreciate you taking time out of your schedule to talk with me!

Lippold Haken: Thank you.

 

About Darwin Grosse: Darwin Grosse is the host of the Art + Music + Technology podcast, a series of interviews with artists, musicians and developers working the world of electronic music.

Darwin is the Director of Education and Customer Services at Cycling ’74 and was involved in the development of Max and Max For Live. He also developed the ArdCore Arduino-based synth module as his Masters Project in 2011, helping to pioneer open source/open hardware development in modular synthesis.

Darwin also has an active music career as a performer, producer/engineer and installation artist.

Images: Haken Audio, Lippold Haken, ContiuuCon, ECE Illinois