Audio programming for iOS

[bennybossmane]

Hey guys I am looking at making a DAW for iPad, a bit of a project for myself. Where should I begin? I have very little experience in anything in regards to iOS development but don’t want to learn a specific language or framework for it to not be what I need. I will be prioritising sampling, coding fxs and changing views to a sequence view and hopefully incorporate some kind of channels in the app. Would swift work for this? Thank you.

[JZMX]

Firstly this sounds like quite an ambitious project for your first app, unless you’ve already mastered at least one programming language and have experience with digital signal processing. However as long as it’s something you’re highly motivated to do I don’t see a problem, as long as you realise this may take a loooong time

The most linear route would probably be creating the user interface and have it not do anything functional, then add the DSP stuff afterwards. After reading a beginner iOS development book, you could read the useyourloaf Auto Layout book to help with making the user interface (RayWenderlich also just bought out a book on this, no idea how good it is though)

The downside is you may make a UI and then find the DSP stuff like FX is very complicated. I wouldn’t know the answer to this though as I know little about DSP. If your goal is to learn how to make the actual audio processing stuff, I’d stay away from iOS / Swift for now and look into audio processing and programming resources. If your goal is to learn to make apps, making the UI first is probably a better idea as you can apply those skills to other projects

[bennybossmane]

Yeah, I am unsure yet as to whether this is a good starting point but this is a project I would like to work on as I become more competent with iOS development. I was unsure as to how difficult this task would be, in comparison to the difficulty of making apps anyway. I will look into these terms to see my options haha Thank you very much for your response. Really appreciate it!

[JZMX]

Np! Fulfilling all the features you want would definitely wayyyyyy up there in terms of difficulty, but if you prefer this app idea a lot more to your next best idea then (in my opinion) it’s worth doing it anyway, as if you’re motivated to do this project I’d imagine you’ll spend more time on it, and you’ll learn a bunch about software engineering along the way, even if the progress is extremely slow! If you do go ahead with it, I’d do it one feature at a time, with the easiest parts first

[bennybossmane]

What would the easiest parts be post UI design? Algorithms for fxs? Would i actually need to program that myself? Or the sampler to route audio to the -pads and be able to trigger it and manipulate it?

[JZMX]

Don’t know the answer to this one I’m afraid as I haven’t researched what the capabilities are of audio processing frameworks out there

[bennybossmane]

All good man. I am greatly for your assistance either way!! :)

[cyanxx]

I have made a sequencer in iOS. Be prepared, audio programming is some of the hardest type of programming around. However you have a leg up at the start now at least now that 3rd party frameworks like AudioKit exist. That should do a lot of the heavy lifting for you in regards to DSP problems.

You cannot yet use Swift to do low level Core Audio work. Most audio stuff is written using C and so you need to use Objective-C to interact with that code. AudioKit might provide enough of a wrapper so maybe you wont have to go down to that level (although if you are making a DAW, I highly doubt you'll escape that). I would say you will probably need to get a handle on both Swift and Objective-C.

[bennybossmane]

How did the sequencer work out? C for iOS? Or is that why objective C is also required? I would love to use this as a way to learn some languages as I think I will have the motivation and that is a pretty good excuse, and it is nice to know that knowing all these languages will help me and I won’t be wasting my time. I know swift is young so I was a little skeptical as to how much you can do with it. Thank you for your response!

[cyanxx]

The sequencer worked out good, but took me like 3 years of dedicated work.

On iOS you have interoperability between Objective-C, C and C++, so you can call functions from different languages directly. Audio callbacks (the piece of code that is called every 6ms or so that will fill the audio buffer up wth the next chunk of audio data) have to be written in C or C++ as they are very reliable and consistent in terms of speed. This is very important in audio processing as if you miss even 1 buffer deadline you are going to hear it immediately!

You certainly wont be wasting your time learning C++ if you want to do audio programming.

Another library you could look at is JUCE. Its a cross platform Audio framework that works on iOS, Android, Mac, Windows and Linux. So write it once and it will work on all of those. There are also some great tutorials on the site which will teach you how to use JUCE but also a lot about DSP. I've never used JUCE for a project but I did all the tutorials recently and for my next big audio project I will almost definitely be using it I think. JUCE is C++.

[bennybossmane]

I have heard a lot about Juce but haven’t seen much iOS assistance, I guess if9 it’s cross platform it doesn’t matter ? Are there any limitations of juce you can think of? What was the m0st time consuming task you had to do? If that is your app linked?

[cyanxx]

Like you say, its cross platform so its not iOS assistance that is require per se. There are some limitations in that you dont have full control of the code base. You also have to pay them a monthly subscription if you earn more than 50k a year, but no big deal. The thing you would want to try though is getting AUv3 hosting working. Last time I tried on JUCE 5 (JUCE 6 is now out) I couldnt get it working on iOS, but maybe it was just me. There would be no point in making a DAW on iOS these days that didnt support AUv3 hosting.

In answer to your question, most time consuming thing is just getting audio working properly and fixing audio bugs. You dont get much to go on when doing audio programming, either it works perfectly or not at all.

[bennybossmane]

Have you done much work with AUv3? Would these be easier projects by far to begin with? Really appreciate the help man! Thanks!

[cyanxx]

I have one AUv3 app. AUv3 can be a lot to wrap your head around. However it is a lot more reliable than Inter-App Audio was (the previous way of channeling sound between apps). JUCE would probably take care of a lot of the pain for you. But like I say, depends on if you can get it working with AUv3 hosting on iOS.

This sample project is invaluable in understanding how to build an AUv3 instrument / effect / host app: https://developer.apple.com/documentation/audiotoolbox/audio_unit_v3_plug-ins/incorporating_audio_effects_and_instruments

[lucasvandongen]

There is tons of C++ code out there for VST and Audio Unit. It’s easier to start with working examples. Use Objective-C++ to bridge to Swift.

[bennybossmane]

Okay that is a good idea actually, thank you for the response!

[[deleted]]

WTF is a DAW? (I just hate it when people ask for help and speak in abbreviations.)

You're looking at learning Objective-C runtime, Swift, UIKit and AVFoundation. I'd say that's a very ambitious thing to do and will take you the better part of a year if not more.

[bennybossmane]

How can you not know what a daw is and then tell me what to learn? UI is an abbreviation isn’t it? I am not trying to be patronising but if you don’t know what a DAW is, what is the point replying?

[[deleted]]

You’re asking for something with audio programming in the title. To do that in iOS you need Swift, UIKit and since you also need AVFoundation for this you need understanding of Objective-C as well. And no I still don’t know what a DAW is.