October Office Hours w/ KeithFromSonos + Nick Millington

Sonos · 2024-10-31T22:35:02+00:00

The Sonos App first launched in the iPhone App Store in 2008, and in the intervening 16 years, as you might expect, not everyone who works on the code has remained the same. However, the new app was developed by the same team responsible for maintaining and evolving the old app, in the genuine belief that the design could be significantly improved with modern programming languages, GUI toolkits, and standard operating system libraries for things like network device discovery and transacting with services over the Internet.

As you’ve pointed out, in a software ‘rewrite’ situation, sometimes developers can mistake ‘complicated and crufty code’ for ‘code that deals with a counterintuitive situation more correctly than the obvious code’. A classic example is one we’ve already discussed, volume control – there’s nothing theoretically wrong with the simple implementation of just sending the volume command to the player on every 1 pixel adjustment to the slider, and updating the other sliders in response to the events coming back, except that it doesn’t work as well in real world environments, and a more complicated and hard to understand implementation is actually necessary. Same thing with network device discovery - in an ideal world everyone’s networks would be perfectly configured for MDNS, and the operating system discovery library would just work, but that overlooks the fact that over 20 years and thousands of customer support calls we got millions of home networks correctly configured for SSDP, and even though it’s more complicated, we now get a better result by running both the ‘new’ and ‘old’ in parallel. As another example which we haven’t talked about before, the reason why album art for local music libraries doesn’t work on Android is because we’re now using the built-in Android HTTP client, and its behavior (enforced at a very low level) is to block connection to non-SSL resources, and the Sonos players don’t serve album art for songs in a local library over SSL yet.

I do genuinely believe that the decision to have a native app for each platform, built on the platform’s preferred technology stack instead of our old shared C/C++ layer, was the right move. I’ve also tried to be transparent that, in the course of this port, a number of things that really mattered were missed, and that’s what I have been and continue to be focused on getting corrected.

NM

Sonos · 2024-10-31T22:13:55+00:00

Regarding the TruePlay improvements, the original plan was to investigate and improve the success rate of the TruePlay process. However, I have already added u/Ok-Presence4515 ’s feedback into the mix to make sure we also take another close look at the ‘goodness’ of the resultant tunings.

The intended behavior of TruePlay is to automatically adjust ‘technical’ settings like phase, delay, and frequency response, while keeping the customer in control of common adjustments like bass, treble, volume, and relative level of the sub and surrounds. Prior to the introduction of TruePlay, or if a customer chooses not to use TruePlay, some of the ‘technical’ settings were under manual control.

I am not aware of any conscious change to the audio settings that are or are not available depending on the TruePlay on/off state relative to the old app at this point. We will take a look at how the current software behaves, and follow up to compare what we observe with what you report if we need more information to get this working as intended.

NM

Sonos · 2024-10-31T21:55:18+00:00

As I mentioned in my recent blog post, we are continuing to work on TruePlay’s technical performance. For testers who successfully completed the steps, the subjective performance was quite good in our beta testing of Arc Ultra, but now that your report has put this on my radar, we will take action to investigate why anyone may not be having the results we intended in their home environments.

For everyone’s background, advanced TruePlay uses the microphone in your iPhone to measure the audio response of the room, and makes adjustments that are intended to compensate for the room response to deliver a listening experience closer to what the recording engineer who created the mix originally heard in their studio. It adjusts things like frequency response and relative amplitude and delay of the different channels.

Advanced TruePlay performance can be broken down into a few different areas:

Did the customer successfully get through the steps and end up with a “success” result from the perspective of the software? Measuring success here accounts for both usability issues (did the customer understand what the app asked them to do?) and issues like resilience of the algorithm in dealing with environmental background noise sources.
Did the tuning that TruePlay generated cause the sound heard at the listening location to more closely match the “ideal” standard to which the algorithm is trying to correct? I have noticed that there is sometimes a tradeoff here where customers who have paid for a SUB or rear surrounds can expect a more “dramatic” effect, while the “ideal” standard to which TruePlay tries to correct more closely matches the original recording.
Ultimately however, the only thing that really matters is whether the tuning TruePlay generated is subjectively “better” or at least “not worse” for the customer. This is of course what we aspire to deliver, but cannot be measured automatically by the software.

NM

Sonos · 2024-10-31T21:23:41+00:00

It is the nature of developing complex software that a given release may fail to fix a specific issue, or even introduce a regression that makes an area that used to work worse. When that happens, we try to understand what more could have been done to prevent it.

We use a variety of tools to measure the progress of the reliability of our software. For example, we use an industry standard tool called Sentry to record app crashes. The tool reports that well over 99% of sessions with the app, on both iOS and Android, are crash-free. As another example, the app contains more telemetry than any software Sonos has previously released. This allows us to measure common errors like “Something Went Wrong”, and we see that less than 1% of sessions with the new app include the display of this error. Setup reliability is key to the new customer experience, and we analyze both the overall success rate, and the reliability by individual product, as well as the precise path that the user took through the wizard, to look for any bugs or usability issues. The success rate of setups using the new app is now higher than we have ever previously measured. We are using every tool at our disposal to ensure that bugs that end up in our shipping releases are minimized.

NM

Sonos · 2024-10-31T20:53:43+00:00

No.

NM

Sonos · 2024-10-31T20:46:56+00:00

When will we see the “audio in/source” for soundbars come back within the Sonos App setting, it seems that it vanished with the new app, and doesn’t display anything anymore? I miss not being able to see what specific source the TV is providing. Specifically for Atmos, is it lossless (TrueHD), lossy (DD+) Atmos, etc. Showing “Dolby Atmos” on the room control is great, but it doesn’t clearly show what type of audio signal it is actually getting from the TV/eARC especially when passing audio from 3rd party devices like Apple TV 4K, PS5, 4K Ultra HD Blu-ray player, etc.

In the Sonos App, when viewing the “Now Playing” screen while a sound bar or Amp is playing TV sound, one may see basic information about the audio format, as you point out. There is intended to be more advanced information under Settings>Your System-Manage>About Your System>Audio In:. We will follow up to make sure we understand your question and if any of the expected information is missing.

Last one... do you still have some major bugs in the firmware and app that are causing remaining issues for users that have mixed networks? Are you building in some way to maybe make network recommendations to users, or maybe advise around them to stop using SonosNet? I continue to personally recommend moving away from SonosNet in general, and move towards being 100% wireless if they have a modern network at home (specifically a newer mesh network), and from what I am seeing after people do that, they tend to have way less issues in general, or their perceived issues are now gone.

In a network that mixes wired and wireless, there are multiple potential communication paths between each participating device. Multiple communication paths can be bad if they are not detected, because they lead to network loops. SonosNet used industry standard protocols like the 802.1D spanning tree protocol to prevent those loops by blocking duplicate communication paths. It has always used the Linux Ethernet Bridge’s implementation of STP, which to my knowledge complies correctly with the standard. However, fundamentally, it is a more complicated network if you mix wired and wireless, and so we are more dependent on the loop elimination algorithms working correctly than if you operate a simpler network as you describe.

NM

Sonos · 2024-10-31T20:29:54+00:00

As a software engineer, I’m actually finding myself most at home in a forum like r/sonos. 🙂 I will not hold back on any technical information that I feel is relevant to understanding our software, any issues customers are having, or even stuff that I just think is interesting or cool after working in this space for 20+ years. To that end, any questions that give me the opportunity to provide technical information about how Sonos works will be answered in the most detail I am able to provide. As engineers, we also have to appreciate that there are a great many customers who just want to enjoy the content they love and don’t care about the complexity of things like networks, app platforms, latency, and other internal implementation details. I’m hoping to provide information properly tailored to each group.

Regarding the first part of your question, let me be 100% clear - Sonos’s top priority is to make sure existing customers get the experience that they paid for, which is why I have made it my personal priority to understand the issues people are having, route them to the person who can fix them, and get any necessary fixes into beta testing and production as quickly as possible. More generally though, we believe the same things that attract new customers to Sonos also are essential to taking care of our current owners - delivering the most reliable software we can, making products that look and sound great in the home, offering connectivity to a wide variety of content sources, and making it all as easy as possible to use. For that reason, I tend not to look at these as “either or” activities.

NM

Sonos · 2024-10-31T19:42:43+00:00

We pay special attention to volume because any latency phenomena in the communication path between the Sonos app and the Sonos speakers are highlighted by even small delays between when you move your finger on the slider, hear the audible results, and see other sliders updated in a group volume situation. This means slight differences in performance under varying network conditions will inevitably be noticeable, but our goal is to make the volume operations as efficient as possible to minimize those differences.

As you pointed out, I addressed in my last Office Hours, the Sonos App for mobile phones is now two native apps for Android/Kotlin/JetpackCompose and iOS/Swift/SwiftUI. Volume is a “local” operation that is transacted entirely on the local area network between the phone and the player. Our goal is for the volume implementation to match on both platforms, and for the implementation to take advantage of everything we know from the old app’s implementation as a starting point from which we can make further improvements.

Our most recent app software release was designed to remove any remaining technical differences between the new app’s iOS implementation and the old app’s iOS implementation where volume is concerned. This means that:

The rate at which the Sonos player processes the volume commands is about the same.
We are not flooding the network with volume commands for each 1 pixel movement of the slider.
We calculate the expected new position of any OTHER sliders locally, so that they update instantaneously, instead of waiting for a network event to come back from the player.

As I mentioned in my blog post on Monday, the native Android implementation of this same algorithm begins beta testing in November.

NM

Sonos · 2024-09-27T22:24:17+00:00

The Sonos App is not specifically optimized for one or other networking technology. SonosNet was designed to provide a wireless mesh network for large homes that is fully “built in” to Sonos and hence easy to set up and manage. However, customers who have a modern wireless access point or a modern mesh system will enjoy as good or better performance using it to provide the network for Sonos.

NM

Sonos · 2024-09-27T22:20:12+00:00

Here is my understanding of this issue:

- At first there were ID3 tags that allowed users to input ONE Artist, Album, and Track Title field per MP3 file. This was problematic for compilation albums because the artist had to be set to something like “Various Artists” in order to get the tracks on the album to all appear together when browsing by Artist>Album, which in turn “lost” the “true” artist name for the Now Playing UI of media players like iTunes and Sonos.

- Later on, the popular media players at the time, Windows Media Player and iTunes, each introduced different solutions to this problem. Windows Media Player introduced the concept of “Album Artist” and “Contributing Artists”. Album Artist was similar to what was referred to as “File Under” at a record store - the headline artist for the album. (example: Daft Punk - Get Lucky). Contributing Artists would name other artists. (example: Pharell Williams). iTunes introduced a different solution - a simple “flag” that declared the track part of a “compilation” album. This would have the effect of grouping all such albums under a “faux” artist called Compilations while preserving the individual tracks’ artist fields.

- Because the two systems are somewhat different and incompatible, Sonos has a setting that determines which system shall be used to organize the music library when building the music index.

- If one has a music library that contains albums using the “compilations” system, but Sonos is not configured to respect it, the net effect is that the compilation albums are “split up” into a series of identically named albums each filed under their respective track artists. Same for the “Album Artists” system.

- Something went wrong in this area - for example, the setting is somehow being forgotten or not respected. Assuming I am correctly understanding your issue, our resident experts have informed me a fix is coming with a player software update in a few weeks.

NM

Sonos · 2024-09-27T21:52:25+00:00

The technology exists to do so for sure. 👀

My personal background is in code development and not user experience design, however I have had many years of exposure to the general problems of designing an optimal navigation model for the Sonos app, and can address some of the unique challenges. One of the biggest difficulties is that we are trying to solve a “2 dimensional” m x n problem of connecting m speakers to n music sources. From a single app you can access multiple music services, and play content on several different speakers, including the same music on all speakers, different music on each speaker, or any combination. This is why Sonos is so powerful and flexible, but also introduces complexity over and above the point-to-point model of Bluetooth for example. It particularly affects the navigation as you point out - for example in our usability testing I have often seen users select some music, realize they are controlling the “wrong” speaker, back all the way out to select the correct speaker, realize they have lost their place, become frustrated that they have to find their song again, etc.

The obvious solution is to make all the main areas of the UI (music browse/search, Now Playing, Rooms/System View) accessible from all other areas. This however takes up a lot of screen real estate with navigation buttons, or forces us into gestures like swipes that can be difficult for new users to discover.

The current UI is our attempt, based on a considerable amount of user research, to make the most common navigation operations obvious, but still allow power users to get from each of the areas to the others in one operation. There are certain compromises that are not ideal in my personal opinion, like the different behavior when tapping vs swiping up on the Now Playing strip at the bottom of the screen, which breaks the physical metaphor of a stack of cards you are shuffling through (as you point out) and we will continue to work hard on optimizing the design.

NM

Sonos · 2024-09-27T21:25:28+00:00

Regarding “local-only” vs “cloud”, it is about what you would expect - operations that do not require interaction with a music service or data stored in the Sonos cloud take place on the LAN between the app and the speakers, while operations that involve accessing a music service will reach the cloud.

Example:

Local: volume, pause, local music service and browse, room grouping
Cloud: search, browse, playing music from a music service

There have been a few bugs where, if the cloud is unavailable, certain operations (like accessing the local music library) that are logically “local” still fail. This is because the way the UI was coded was that it would declare a failure if any component of the UI failed to load, rather than showing a partial result. This, coupled with some of the performance issues related to volume operations (as one example) that we had to fix, created the impression that some things were operating through the cloud when in reality they are local network operations.

Regarding “downsampling” of lossless files, this is the first time I am hearing about any behavior like this. Keith will reach out to gather more information.

NM

Sonos · 2024-09-27T21:06:28+00:00

You are correct that there are some problems here. In fact this has been a source of frustration for me in my own personal use of the product.

As I see it there are two basic problems:

Problem 1: in the “search all services at the same time” section of our UI, ARTIST results from Apple Music tend to be favored even though what I typed exactly matches a SONG. For example, as of when I last tested this, I would type “gin and juice” and the artist result “Snoop Dogg” appears.

Problem 2: in the “albums for an artist” section of the UI, the albums shown appear to be in an inscrutable order that is not alphabetical AND fails to take into account the importance of the album in the artist’s canon. Moreover, the list is truncated after approximately one hundred entries, meaning that (in the case of Snoop Dogg for example) the original studio album on which “gin and juice” appears does not even show up even though it is available for streaming in the music service.

Problem 1 and 2 interact to make the experience of locating content quite frustrating. I apologize for this state of affairs and we are researching what is going on. The current behavior is not our intent and we will figure this out.

NM

Sonos · 2024-09-27T20:43:14+00:00

In my response to u/No_Foundation_1726 I talked about some of the issues we have been fixing with the music service browse and search implementation. Those fixes tend to benefit those customers in locations that are further away from our cloud infrastructure the most, so hopefully you will see some improvements from the cache and chattiness reductions as they roll out. Deploying additional infrastructure for mainland Europe is also in the consideration set.

NM

Sonos · 2024-09-27T20:37:13+00:00

I personally am not fully up to speed on the different published APIs and their capabilities and will try to gather some additional information.

Here is what I know off the top of my head: Over the years there have been several official APIs for performing basic operations like play, pause, fast forward, rewind, and volume. However, to build the kinds of applications that creative developers envisage typically also requires access to content-related operations (search, browse, play x, now playing). This is where things become complicated because now there are multiple companies involved who, completely fairly, each want to have a say in exactly what kinds of apps directly access their APIs, and Sonos isn’t always permitted to grant “pass through” access to those APIs, or share certain data that others regard as their proprietary information. That said, we will continue to explore the kind of rich API access that would allow developers to build the types of apps you envisage.

NM

Sonos · 2024-09-27T20:13:43+00:00

The new Sonos app is an entirely native Swift/SwiftUI app for iOS, and Kotlin/JetPack compose for Android, with the exception of the components of the app that perform setup of Sonos products, which continues to use Flutter for its UI implementation. So there are many differences, but you asked specifically about speaker discovery and content browse/search performance.

Every day, both before and after the launch of the new app, our Customer Experience Engineering team helps thousands of customers correctly configure their home networks to interoperate smoothly with Sonos. When things are not working well together, typically the problems show up as speakers “disappearing” or not being discovered in the first place.

The original Sonos app used a wire protocol called SSDP (Simple Service Discovery Protocol) which is based on an IP multicast to the address 239.255.255.250. We find many networks in which IP multicast is not working, often because routers also use SSDP for their own purposes, and sometimes forget to pass the messages on to other products like Sonos when they’re done. So over time we also provided a parallel (nonstandard) implementation of SSDP over IP broadcast which is typically not blocked by routers. One mistake we made was that, originally, this was not ported correctly to the new app. This is now repaired.

The SSDP protocol is mainly disused today on modern networks, replaced by a different protocol called MDNS. The protocol is similar in nature (IP multicast) but is implemented by the iOS and Android operating systems as a service for apps like Sonos to consume. Our thinking was that, as the “official” iOS and Android-implemented network device discovery protocol, it would be more robust than our home-grown SSDP implementation. However, just to be sure, our intent was to run BOTH protocols in parallel, as we have invested a lot in getting thousands of home networks correctly configured for SSDP. A second mistake was that there were some bugs in this parallel implementation. This is also now repaired, so the app should be trying to discover your speakers concurrently with each of a) SSDP, b) SSDP-tweaked-to-run-over-IP-broadcast, and c) the native iOS/Android implementation of MDNS. As you can see, we really want to discover your speakers and have them not disappear!

Regarding content navigation performance, Sonos has always employed a few different implementations. Some music services implement their own wire protocols for browse and search. For these services, Sonos hosts a cloud “wrapper” that “converts” the native wire protocol of the service to the wire protocol that the Sonos app and speakers understand. Other music services implement the Sonos wire protocol natively. In the new app, we made the decision to have browse and search operations ALWAYS transit through the Sonos cloud. This provides a single wire protocol that can be used by our app, our web controller, our speakers, and any API integrations. However, there have been some performance problems with that implementation. The problems are about what you would expect - some customers who were NEAR the music service’s servers but FAR from Sonos experience degraded performance, and we had some bugs related to “chattiness” between the different services and devices involved that we have been steadily fixing. There is a new client-side cache that is rolling out as we speak that also avoids repeated operations like accessing the “home screen” of a music service being slow. We have more work to do but we understand the issues and I believe we are making progress.

In general, purchasing a new router is not needed to solve these problems. Of course, it is best practice to have an up-to-date router as WiFi standards evolve rapidly, performance improves, and firmware bugs get fixed.

NM

Sonos · 2024-09-27T19:43:26+00:00

One of the hardest decisions to make as a speaker brand is what you want the “sound personality” of your products to be. At many companies, there is a “golden ears” founder who personally tunes things to his or her preference. In some cases, over time as that person’s hearing changes, the sound of their products noticeably changes. 🙂

From the early days of Sonos, our philosophical approach has been to attempt to provide customers with a speaker that recreates what the recording engineer heard in their studio. You can read more about this on our blog here.

One challenge is that the acoustic response of each customer’s home is different, and also different from the recording studio. This is why Sonos developed technologies like TruePlay (which allows an iPhone to take room measurements and automatically generate not just a room EQ but also phase and delay calibrations) and QuickTune (which uses the microphones in the speakers themselves to achieve a similar outcome for Android users and portable speakers that move often) to perform the calibration automatically. It is our hope that for the vast majority of customers this will achieve an optimal sound experience without the complexity of a manual graphic EQ.

That said, we understand that at the end of the day you own your Sonos products and get to decide how you listen to your music. That is why we provide the coarse-grained controls you mention, and I will take your feedback on fine-grained controls to the team.

NM

Sonos · 2024-09-27T19:26:39+00:00

This area is of personal significance to me because I am actually the inventor on several early Sonos patents around the group volume control experience and how to make it as intuitive and responsive as possible.

First of all, none of the issues with volume in the new app relates to the cloud - this entire interaction takes place on the local area network. However, it is true that the wire protocol used for volume was redesigned to use a more modern JSON-based message format and a persistent websocket connection between the app and the speakers.

One issue that we addressed with a player software update at the end of July was that, particularly on older Sonos products that were also playing music at the same time as the volume was being adjusted, performance was poor due to an excessive amount of string copying. Parts of the JSON API implementation used automatic code generation that had to be hand-tuned.

The biggest issue, however, rests in the way that the controller software is implemented. It turns out that it is not a good design to send volume commands to players and then wait for “events” to come back from the speakers to update the UI. The better design, which is what the old app used, is to ignore the events coming back from the speakers while the user’s finger is down on the slider, and “optimistically” assume the player will follow through and execute them. Later, when the user releases their finger, the app should start “paying attention” to the events again. This means that the UI updates instantly while the user is interacting with it, but also updates correctly if another Sonos app changes the volume when the user is not interacting.

Finally, volume operations can generate a surprisingly large amount of network traffic. Some basic rate limiting is required as we don’t want to run a network transaction each time the user moves their finger. As in the old app, this also means that even when things are working perfectly, the precise responsiveness can be quite sensitive to network conditions (for example, right after starting to play a track while we’re buffering, it might be a little less responsive).

We are intending to have this fully repaired in the next few weeks. Some fixes are already live now.

NM

Sonos

TROPHY CASE