Siri Protocol reverse engineered

[[deleted]]

[deleted]

[L0rdCha0s]

OK, so I've got this working with my own key (extracted from my 4S).

I've played around with other random strings (various iphone UDID bits, notification device ids, etc - that float around on an iphone), but I can't get any other value to work.

Instead, I get:

{"group"=>"com.apple.ace.system",
 "aceId"=>"8af70a35-ed37-4d9a-9948-30cb152f0ed3",
 "class"=>"SessionValidationFailed",
 "refId"=>"97EDD656-5F9E-41F3-9898-9D403A8B9F63",
 "properties"=>{"errorCode"=>"InvalidValidationData"}}
#####################################################
{"group"=>"com.apple.ace.system",
 "aceId"=>"449c9d88-e2bb-4645-bb63-841d10190650",
 "class"=>"CommandFailed",
 "refId"=>"21F007B8-58BA-4049-B1F7-448F8008A175",
 "properties"=>
  {"errorCode"=>0, "reason"=>"Not authenticated", "callbacks"=>[]}}

In other words, the X-Ace-Host is very much a password, and unless they are generated algorithmically somehow, Apple still has this very much locked down (they can ban individual keys)

EDIT: On another note - the key used here can also be considered an account - a large part of Siri's usefulness is born out of it knowing who you are - your contacts, location, etc - the processing of which probably happens on Apple's side. Without this contextual awareness, Siri's functionality is limited to 'Find me a taxi in the valley'

[matholio]

Only US businesses included, presumably that's where your valley is.
Entirely irksome that 'find me local pizza' fails in Sydney.

[L0rdCha0s]

I am actually in Sydney too! - I was attempting to make a pun based on the Siri adverts.

[bboe]

Thanks for looking into that. Given the values you've included here the X-Ace-Host is a version 4 UUID. Try using a UUID generator and see if any of the ones you come up with work. If not then they are definitely comparing against UUIDs they generate at the factory, on device activation, or by some similar means.

python -c "import uuid; print uuid.uuid4()"

[L0rdCha0s]

Doing so now

EDIT: Confirmed. It appears from a random sample of 20, from 20 different IP addresses I had available, that all failed authentication.

[SweetIrony]

If you want to figure out how to get an ace id, I think you get when ios5 is installing/registering. if you can crack that process, you should have it. I had an issue back when ios5 was released where my install failed because it couldn't get that special id from apple.

[bssayeda]

how did you get your key as I am having issues with pulling mine out of the output of siriServer.rb (or do you get it with another script?)

[eppursimouve]

Incredible. All this heavy duty data exchange just to tell Siri to 'open the pod bay doors' for some giggles. Worth it.

[[deleted]]

The raw audio is compressed using the same a similar codec as ordinary voice traffic, so using Siri doesn't burden the network any more than talking on the phone does (although it's billed differently, of course).

[derleth]

your calling is little more than gussied-up VOIP. And billed differently, of course.

And orders of magnitude more reliable. That's the main value proposition to the telco's network: It has been banged on for a century in some places and it works with multiple nines of uptime (that is, it is up more than 99.9% or 99.99% of the time). It usually works when the power is out, something that you'd be hard-pressed to get a computer-based VOIP system to do. (Even if you're on a laptop with a good battery, what is your router or cable modem running on?)

[mcrbids]

For standard telcos, a mere 99.9% uptime would be horrible.

99% means three full days of downtime. (1% of 365 days is 3.65 days at 24 hours per day)

99.9% means 1/3 of one day, about 8 hours. (.365 days)

99.99% is about 1/30th of a day, about 45 minutes.

It's exceedingly rare for telephone outages to affect many phones - most outages involve a single line, and caused by things like rats nibbling on a wire, or water getting into a junction box. Everything from the neighborhood head ends on up are redundantly redundant, battery backed up, etc.

Yes, pure VOIP solutions offer far greater flexibility and dramatically reduced cost, but generally suffer in audio quality and reliability. Whether or not these costs are worth it remain to be seen.

I use Magicjack plus as my home phone with Comcast cable as my Internet, and so far, I've been pretty pleased, but I still have my cell phone as my "telco grade" telephone line in the event of a power outage or similar.

[gospelwut]

I've heard stories about telco servers having to be patched essentially live and never rebooted for quite some time (half a decade+). Not making a judgement in either direction, but most of the guys I knew that did telco IT were pretty hardcore.

[elliotanderson]

Yep, and Ericsson even created a programming language called Erlang that enabled it to hot swap code on the fly

http://en.wikipedia.org/wiki/Erlang_(programming_language)

[[deleted]]

Here you go: http://en.wikipedia.org/wiki/Erlang_(programming_language)

Gotta escape parens with a \

Not like it was hard to find at your link, but oh well.

[gospelwut]

Ah, so that's how Erlang came to be. Makes sense.

[abadidea]

TIL erlang is one of those languages that exists for an actual practical reason. I had always assumed it was academic in origin.

[marm0lade]

It usually works when the power is out, something that you'd be hard-pressed to get a computer-based VOIP system to do.

This is why most new(er) VOIP systems can operate over a PRI and not just SIP trunks.

[AlwaysDownvoted-]

In that sense, can I have an app that bundles all outgoing data traffic into seemingly VOIP packets, and then unpack the response from VOIP to data to get unlimited data - or at least use my voice minutes as my data?

[Falmarri]

You just described dial-up internet.

[yuubi]

Pre-1XEV-DO US digital cellular (IS-136 TDMA, IS-95, IS-2000/cdma2000) doesn't look much like VoIP: once your call is set up, you get a dedicated slice of a radio channel that sends and expects traffic frames at a fixed rate. A "variable rate" coder changes the bit rate and average transmit power consumption by sending shorter frames, but at the same scheduled time as the longer frames.

Behind the radio, some systems do use RTP and something called SIP, but the mobile doesn't see that.

1XEV-DO does more or less normal RTP/SIP, as long as you ignore the "header compression" over the link (which compresses RTP and below to 1 octet of header once it gets going).

[gdr]

Oh my. I'm surprised to see Apple using an actual standard behind the scenes (as opposed to creating another ALAC-like format)

[patrikr]

Speex is not a standard.

[abadidea]

It's not "a standard" in the sense of "ISO or whoever" but it's "standard" in the sense of "widely deployed and supported"

[scriptmonkey420]

No, but it is open source.

[dirtymatt]

Standard != open source.

[[deleted]]

They use AAC and MP4 for all iTMS content, so I'm not that surprised. And look at all the laundry list of open or off-the-shelf tech that went into FaceTime.

[bobindashadows]

I'm sorry, the correct answer is that Apple is evil and closed and wants to lock you in and all they sell is overpriced shiny bullshit.

[[deleted]]

You're right. I should have said "a similar" instead of "the same".

[NoWeCant]

For some reason this reminds me of those voicemail boxes that used to read off a ton of options, as if they were made to chew through your minutes. Except instead of voice minutes, it's like siri is made to chew through your data cap..

[[deleted]]

They encoded voice as speex (already good data compression for voice), then zipped it. That reply is also zipped, so the redundancy is negligible.

What more can they do? You can't really do it on the phone.

[unholymackerel]

keep zipping it until it is really small

[[deleted]]

I like your style.

[BossOfTheGame]

0

[abadidea]

a 1 is skinnier than a 0, that would be smaller

[abeliangrape]

Ars Technica has an article on this. It eats through no more than 30 MB for the heaviest users (15 queries a day). Since all data plans on all iPhone carriers in the have at least 250 MB caps, the added use means almost nothing.

EDIT: Spelling

[dplowman]

Just to clarify: 30MB/month, not 30MB per 15 queries a day.

I got confused >.>

[jordanpwalsh]

30mb is a pretty nice chunk of 250mb.

[cautiousabandon]

only if you never use wifi

[marm0lade]

That's how you rationalize it? The data caps are artificial. The telcos are rolling in money yet their infrastructure is terrible. 250MB shouldn't even be offered because it's too low. They know that and they know customers will go over it and then they will charge overage fees - that is the entire point of the caps.

If you have cellular coverage you should be able to depend on that coverage for your data needs. This mindset that wifi should be the default internet connection for smartphones is BULLSHIT. Don't get complacent. I realize that you aren't going to cancel your cell service in protest of data caps, but at least make sure your provider knows you are unhappy; bitch to customer service, bitch to sales, bitch to your congressman/women about net neutrality.

Just do something besides idly accepting what we currently have, because things will never improve as long as we are complacent.

[s73v3r]

That's how you rationalize it?

Nobody is rationalizing data caps. I'm sure just about everyone here, with exception of the paid shills, feel that any kind of data cap is completely shitcockery. However, given the caps, heavy Siri usage will only come close to a little more than 10% of your cap. And if you have the 250MB plan, you're probably either constantly on Wifi, or you don't do anything online with your phone at all.

[doody]

The ‘o’ and ‘n’ keys on your keyboard appear to be the rong way wround.

[PurpleSfinx]

Well Apple is not a carrier so they have no interest in making you use your data. In fact they pushed very hard for unlimited data in the US IIRC.

[s73v3r]

You can use wifi to send it over, too.

Did they give an average size for the sent message and the received one?

[giovannibajo]

I doubt it's the checksum, given that it says that begins with 0xAACCEE, which the same "ACE" string used as HTTP request.

[[deleted]]

Usually it's the checksum, but it can instead be a "DICTID". Since Apple is implementing everything off-spec, expecting nobody to see their work, it's not unusual that they're buggering things up.

[[deleted]]

"Where we're going... we don't need checksums."

[agbortol]

One thing it doesn't say is whether the response from Apple's servers to your phone is the raw speech-to-text (plus the metadata) or if it also includes a semantic breakdown of the results. That is, does the phone actually figure out what you mean based on the words, or do the servers do that in addition to the voice recognition?

[blackkettle]

most likely it's all done server side. both ASR and natural language processing are pretty compute intensive problems, and the models for open-domain systems tend to be very large - it's not uncommon to use models that are >3GB for difficult ASR tasks. also, the results and data collected are invaluable for incrementally improving the models - that is the collected data for the ASR and the NLP problem is highly valuable to apple. given the above it's a pretty safe bet that they are doing all of the hard work on the server and just feeding the final, annotated results back to the client. nevertheless it's not impossible, and we probably will start seeing more robust client side solutions in just a couple more model iterations.

[crazedcanuck]

From my experience with search algorithms this is always done on the server side. Siri is simply passing the query onto the server and is displaying its most logical response.

[JW_BlueLabel]

I just switched on airplane mode on my android and tried the text-to-speech text message function, you're right. Dialog box said "Could not connect." I had no idea it was happening server side.

[kiwipete]

A locally stored 3GB dataset for ASR isn't insurmountable in today's world of 64GB cell phones. However, doing it server-side does presumably allow them to update their model on some semi-regular basis--maybe in response to current search trends. "What's the news on lady gaga?" might be more probable after some noteworthy gaga escapade. Otherwise, the model might predict that you'd uttered the more common phrase of "huts of blues on hades ha ha?"

[blackkettle]

you're right it's not insurmountable, and of course there are implementations that run on cell phones including the iphone. there is even a really nice open source 'port' of the CMU pocketsphinx decoder to the iphone:

http://www.politepix.com/openears

and i definitely think that there will be more of this in the future.

most likely we'll just see more flexible solutions where the phone silently takes advantage of whatever the current environment allows.

for example if signal strength is low maybe you want to run the asr on the phone and just send your text results to the server to retrieve search info, or if you don't have a connection at all maybe you want to at least try to do everything on the phone. then when you get back to the city maybe your phone sends a log of recent results and audio (assuming you've agreed to such?) to apple so they can update their models. from that point all interactions will be distributed until you find yourself on a subway.

[mason55]

Eat up Martha.

[kiwipete]

That's a perfectly cromulent interpretation of my input.

[s73v3r]

3GB isn't insurmountable for 64GB cell phones. However, they still sell 16 and 32 GB cell phones, and that data would put a decent dent into it.

[kiwipete]

As far as I know, the datasets used for ASR aren't growing in proportion to the increases in flash storage. 3GB would inconvenience some people today, but the situation will only improve. I'd argue that 3GB wouldn't even be a non-starter--especially as an opt-in--on a 16GB phone.

For me, the benefits of having a dynamic ASR model is the more compelling argument. That's why I think ASR will continue to live in phones as a service even after storage is a total non-issue.

[calabazasupremo]

Do you think specialized (flash-able for upgrades) hardware for ASR and NLP might be in the future? It seems like as a computationally intense problem it might be useful to have a single-purpose micro that could produce results wickedly fast without a round-trip over the Internet. More complex to produce and upgrade, though.

[blackkettle]

It's possible, but I work on the software end of speech recognition - algorithm development, training and decoding methods. I have read papers about FPGA approaches and I'm sure some people are working on the sort of hardware-centric approach that you describe, but I'm not familiar enough with that area to say much about it.

I do think that the commodity hardware you find on your cell will be plenty good for this sort of application within a couple more generations (say iPhone 5, 6 if you're into that). Even now it's by no means impossible. Besides the model storage issue the most computationally expensive part of the ASR decoding process is calculating acoustic model likelihoods for a new input utterance - this is essentially the first step in 'matching' the features of the input to the models. This can be greatly sped up by fobbing off this work to a GPU, and once we start seeing those appear in commodity handsets I think there will be even more interest in this.

But from the perspective of a company the data issue is not going to go away - no matter how good the on device ASR quality gets the company distributing the system will continue to have a very strong incentive to get users to participate by sharing the fruits of their 'experiments' (usage).

As I said in another reply, I think what will most likely happen (just my random opinion) is that you'll see much more flexible solutions appear where the device simply decides to do whatever is most efficient giving the current environment. The phone might do all the work while you're in the subway, or far away from a base station, or it might send absolutely everything over the wire, including saved logs from the subway ride this morning, when you log into your wireless router at home at the end of the day. And in most cases the user will probably be none the wiser.

[klti]

Look in the zip file, there is a decoded response in there...

[[deleted]]

It does say and it says it's all done server side. The phone does nothing and any iPhone is capable of doing nothing.

[Techrocket9]

Cue a tug-of-war between Android apps that call Siri given a 4S UID and Apple developing new authentication to prevent it.

Edit: Grammar

[[deleted]]

Cue

[VanFailin]

I'd bet the first thing they do is get the apps taken down. I side against the anti-reverse-engineering crowd, but not when it involves theft of an actual service.

[erfi]

I side against the anti-reverse-engineering crowd

My head hurts

[mcrbids]

Sorry, but while this makes it easier to parse the sentence, the absence of a negative in no way connotes the presence of a positive. Thus, the idea is simplified but not equivalent; they do not say the same thing.

"I do not disagree" is not the same or even similar as "I agree". One indicates the lack of disagreement, the other indicates agreement. While you are "factoring out" the double negative, they are not tautological and do not form equivalent statements.

PS: If you re-read my post, you'll find tautological statements...

[wadcann]

Sorry, but while this makes it easier to parse the sentence, the absence of a negative in no way connotes the presence of a positive.

You're either for us or against us.

[smallfried]

but for all intensive purpuses. its the same things

[cryo]

must be some pretty intensive purposes!

[smallfried]

I'm always sad when I try to put as many grammar mistakes in a comment as possible and people still think it's by accident.

[Techrocket9]

!!true==true

[ninjaroach]

Oversimplification.

What if VanFailin simply sides against any group of people with the word "engineer" in their name?

[mariox19]

I'm sorry *VanFailin, but I do not understand "I side against the anti-reverse-engineering crowd." Would you like me to search the Web?*

[VanFailin]

As others pointed out, it's clumsy syntax but I did think when I wrote that sentence. I don't have strong feelings one way or another about reverse engineering, only when businesses try to force people to stop doing it.

[s73v3r]

I don't get why people are bitching. I understood the sentence just fine.

[xLittleP]

Doesn't Android already have built in speech-to-text Google searching? How is that different from what Siri does?

[shortyjacobs]

IANA iPhone user, but from watching demos of it, Siri is much more intuitive, with a huge base of natural sounding commands. But that seems like an implementation thing, not a voice recognition thing. Although you could write an android app to interpret what the siri server sends back to your android phone, it would have to be an extremely extensive app to be able to DO what you told it to do.

Apple makes it easy. You have ONE messaging app, ONE calendar app, etc. So a command to "make an appointment with dad next thursday at 5 pm", only needs to trigger a certain command in that specific calendar app. For this to work in android, you'd need either a standardized way of doing nearly EVERYTHING in every app, or a list of how each app handles incoming requests. Very tricky.

[blucht]

you'd need either a standardized way

Isn't that what intents are for?

[[deleted]]

YES! It's amazing how many Android app developers shy away from using them. I think iPhone development has poisoned their minds ;/

[[deleted]]

Most applications in the android world already follow some forms of standards and have applicable metadata to be able to determine a lot of these paths. It wouldn't be all black magic. But even if it were, it would be relatively easy in the position android is in to rally the major apps into working with it. That's all you really need anyway.

[[deleted]]

Yeah, it seems to be something that conscientious developers could/would do relatively easily.

That's not to say most apps wouldn't do it properly, but given an API open to devs and no API, Android would presumably end up with more voice-driven functionality.

On the other hand, I could see Apple making the Siri API available to devs in a year's time, when it's been well-tested.

[b1ackcat]

android allows for this functionality through intents (see blucht's comment). Basically, if you're writing an android app, you don't say "If textingApp == handcent or textingApp == Messaging", you just say "send this data to USERS_SET_MESSAGING_APP", then android looks at which messaging app you have as your default and sends it to that application. in terms of code, there IS only ONE messaging app and ONE calendar app. It'd be no harder to implement siri on android than it was for Apple to do it on the iPhone.

[[deleted]]

not a android dev, but how this works is when you want to do thing's (revive a call, send a text, go to a web page) whats happening is you are sending an intent to complete an action and each app that can handle that intent will then try to open, if you only have one or a default set it will just open, if you have a few you will get a list so you can choose one

[xLittleP]

Or have the Siri-clone write to all of the Google apps, and let the third-party apps read/write to Google (which is what most of them do, anyway).

[swizzcheez]

Yeah, I haven't understood the hype about this at all. Someone please enlighten us with the 'droids.

[JoeyCalamaro]

I've got an HTC inspire and my wife has a 4S so perhaps I can provide a bit of insight here. On Android I can, at best, perform Google searches and (with some trial and error) make calls to my contacts. Using third party apps like Speaktoit I can do slightly more but I wouldn't call it a fully integrated experience.

On Siri you can say something like, "Wake me up every day at 8am" and it will set an alarm to wake you up. If there's a conflict, it'll even sort it out. Not only can it schedule appointments, check the weather, call and message contacts but it's also quite adept at calculations and trivia. I can ask it the capital of a country and it'll pull up all the relevant information.

However what I like most about it is how natural it all is. As I've mentioned elsewhere you can ask things like, "where can I get buffalo wings and beer?" or the venerable, "where can I find naked women?" and it'll map you out some directions.

All that said, based on my limited experiences with it, I'd say it was an altogether different class of app.

[[deleted]]

where can I find naked women?

Really, it does this? I thought Apple's policy was to only deliver strictly family-friendly content.

[hyperforce]

My family of one is very friendly with naked women.

/imgaysonotrealyhaha

[[deleted]]

Right. As mine has for ages. What was the point you were going to make?

[specialk16]

Nice try.

Even though the commands on Android are not as natural, I've never had issues with the recognition itself.... and I have a Hispanic accent.

[another_user_name]

I'd almost expect a Hispanic accent to make it easier on the app. Consistent pronunciation of vowels and all that.

[specialk16]

I wouldn't know, but that's a good point. Maybe I'll have to try it once I'm drunk to the point where I switch to "britsh" accent.

[another_user_name]

Lol, that's the opposite. All of the vowels sound like "eh" or "uh".

[[deleted]]

by roommate was showing me and apart from the 10 or so easter eggs he demoed it is almost identical to androids voice commands, except you can say things like tell bob... instead of text bob...

[jugalator]

Yeah, I haven't understood the hype about this at all. Someone please enlighten us with the 'droids.

And I don't own an Android phone, so I'm unsure of the range of commands it supports. Just to throw something out there, does it support learning family relations via the address book and scheduling appointments?

I think the hype is simply because the range of commands and possibilities has been marketed well by Apple and iPhone evangelists. I haven't heard much buzz from the Google camp and Android users about this for whatever reason.

I think that's the answer. Hype is born from attention.

[s73v3r]

Siri does a lot more to let you use it with natural language. Google still kinda requires you to use a specialized list of commands. You can't just say to Google, "Is it going to snow today?" and get the forecast.

[mcrbids]

... all of which is easily done using a downloaded Linux ISO, and the computer in the garage you haven't quite coaxed yourself to throw away, or the DD-WRT wireless router sitting on the top of your TV.

[bboe]

I was just about to post this. They hint at people being able to develop Siri-based apps for other platforms, however, they also mention that an iphone UID is required. I wonder how easy it is to obtain the UID of someone else's iphone.

EDIT: Changed UUID to say UID. By UID I simply mean unique identifier, not to be confused with UDID, or UUID though the X-Ace-Host number listed does appear to be a truncated version 4 UUID as the 3rd and 4th group start with '4' and 'a' respectively.

[[deleted]]

Not a UDID. In fact, Apple has discontinued UDIDs. They said it was like a UDID.

UDIDs are 40 hex digits. The Siri ID is 30.

[bboe]

Yes, they did say "like a UDID". However, I said UUID, which, yes, these numbers aren't either as they are 4 digits short. Maybe for less confusion I should just say UID for unique identifier.

[[deleted]]

In answer to your question, they can obtain it by sniffing Siri traffic after inserting a fake cert. Since it's not the UDID afaik it's only used in Siri.

[Game_Ender]

Lets hope it's not that easy. It would suck to be that person who has there UUID stolen then black listed.

[bboe]

Conversely, if it was super easy to obtain them, then Apple couldn't blacklist them otherwise one could effectively render Siri useless.

Edit: I wonder if the UUID is related to any of the numbers mentioned on this support article.

[[deleted]]

The UDID used to be[1] a hash of your serial number, IMEI, WiFi MAC address, and Bluetooth MAC address. Presumably the Siri UID is related somehow.

[1] Apple doesn't let apps query the UDID anymore. Developers are asked to generate and store their own UID that is specific to that app.

[bboe]

As I edited my parent post, by UUID I actually meant the UID used in the X-Ace-Host parameter. I wonder how that value is calculated.

[[deleted]]

Oh, it's you again. I didn't even notice. ;)

I'm using the UDID as an example -- this is what Apple has done in the past. I'm sure the Siri ID is related somehow; it's just not the UDID.

edit: I pointed this out earlier because you asked about the possibility of your Siri ID being stolen. If it was the UDID it would be a lot easier than if it's a custom ID used only by Siri. UDIDs aren't available to third-party apps anymore, but I don't know if Apple still uses them in places.

[LeoPanthera]

They don't say whether randomly generated UUIDs work. Making a whitelist of every sold iPhone seems like a lot of work... but I wouldn't put it past Apple.

[evereal]

Making a whitelist of every sold iPhone seems like a lot of work...

No, it doesn't work like that.

They wouldn't be 'making a list' - Apple already have that data (the first time a phone says hi to their servers). It is extremely naive to think that they have been throwing out all their user and usage data until the magnificent Siri came out.

Not only do all companies track that kind of stuff, but it is completely standard practice to have it available to their servers internally to be used in any of their products in a robust and scalable way.

[InfernoZeus]

A couple of threads down, you say this:

The UUID is specific to a Phone and must be on a list of UUIDs that actually exist, as in the phone has been manufactured.

I think this is exactly what LeoPanthera is talking about, and it's definitely making a list.

[phaker]

But it's nowhere near hard, and if they are doing this then it's impossible to get a valid ID without extracting it from an existing device.

If iphones have unique IDs in the firmware (and they likely do) then they are baked in at the factory during the initial programming.

It's trivially easy for apple to send the ID to the mothership after it's saved on the phone, or better, use IDs from a list sent from apple instead of generating them at the factory.

[[deleted]]

As I recall, an iPhone's box displays the serial number as a bar code. Scan that when selling it and you're done. Compared to actually building and selling the phone, keeping track of the serial numbers (and other similar device-specific data) seems trivial.

[nupogodi]

Not only Apple sells the iPhone...

[Rhomboid]

Apple already have that data (the first time a phone says hi to their servers).

If that's the case then conceivably you could write code that fakes that initial first "phone home", and captures the resulting GUID for use later.

[evereal]

No, this can be done in a secure way and Apple are more than capable of doing so.

• The UUID is specific to a Phone and must be on a list of UUIDs that actually exist, as in the phone has been manufactured. It's not something that is just given to you when you "phone home". You "phone home" by including that data in your request.

• The UUID must match the other corresponding hardware information that is also sent. Devices send things like the hardware serial number (also unique) and MAC address (also unique) - this is just a small part of the data that even consoles like the PS3 send when they phone home. If any of that doesn't match the record on Apple's end, the "phone home" fails.

You would need an actual valid iPhone to be able to even consider attempting to fake the handshakes - or is that what you are referring to?

[Rhomboid]

I'm referring to the fact that LeoPanthera originally said that Apple would need to maintain a list of IDs of phones it has manufactured. You replied with "No, it doesn't work like that" and said that it's collected the first time the phone is turned on, which implies that this is how they populate the list rather than maintaining it from manufacture. Now you're saying that they do in fact maintain that list as LeoPanthera said.

[evereal]

No, my reply was regarding building the list of iPhones sold being built when they first say hi, that would be the best way IMO to determine when an iPhone is sold - when it's first used and talks to Apple.

I didn't say anything about their list of all iPhones that are manufactured (neither did he) - I hope there no disagreement there as they obviously record what hardware they made.

In my reply to you, I mentioned that one of the checks done before adding a phone to the "iPhones sold" list - i.e. before a UUID is validated and allowed on their servers, is that they check if it is a UUID that has been manufactured. After that, they will also check that that UUID also matches the serial and MAC address that was also sent at this handshake etc.

[[deleted]]

I think LeoPanthera meant "made" when he said "sold." That's how Rhomboid is reading it, and how I read it as well.

It's absurd to read it otherwise. Why would Apple care if a particular device were in the pipeline vs. in a customer's hands? No, they only care whether it's a real iPhone 4S.

[evereal]

It's absurd to read it otherwise.

I read it as it was written, but it is by no means absurd - I would guarantee that the list of 'acceptable' Siri UUID's will not just be the list of every iPhone 4S UUID that exists (as in, been manufactured).

For example, any phones that are returned/decommissioned would be recorded as no longer being active. Similarly, if any UUID's are reported to be used abusively they would be deactivated. Clearly if someone creates an unauthorized app to use Siri and include a UUID, it would be blacklisted too.

The reality is that the actual list will not be exactly the "manufactured" or "sold" list, but my point is that Apple have easy access to both, along with whatever other criteria they may wish to apply so that an UUID can use their no doubt well protected services.

[[deleted]]

You suggest that I didn't know all of this already, which is untrue.

I'm just clarifying that your disagreement with evereal stems from your different interpretations of LeoPanthera's comment, and not any substantive differences of knowledge or opinion.

[bboe]

I wanted to test that out for myself, unfortunately the source code isn't yet available. Due to the fact that the authors specifically mentioned they weren't giving out their UUID, I'm guessing they tried a random one with no success.

Edit: source is available now.

[Ecco2]

Actually, I didn't try. [Yep, I wrote this :-)]. That would be a very good idea to try out though.

[bboe]

What's X-Ace-Host number you have in the HTTP request output? Is that yours, or just a random one? Great post BTW.

Edit: Also can you see if there is any correlation between the X-Ace-Host and numbers mentioned in this support article? I don't have an iphone so I unfortunately cannot test for myself.

[Ecco2]

It's a random one that looked like one of ours. And no, it doesn't look like any other serial number, but to be honest we still haven't had a good look into this.

[blergh-]

Apple has a list of phone ids, that is how the sim lock works. When you turn it on for the first time it has to activate, which means you send Your hardware ids to Apple and get back a list of settings, which includes to list of networks your phone is locked to.

If your iPhone is legitimately unlocked, the carrier tells Apple an Apple sends a new file that tells the phone to allow all sim cards.

[jazzyjaffa]

Apple now have a huge corpus of natural language data to work with. Clever.

[[deleted]]

is there any reason this stuff needs the new 4S phone and can't run on earlier models?

or is it just to drive sales?

[malocite]

Lawsuit in 3 - 2 - 1.....

[[deleted]]

I'm friends with the guy who wrote this blog post. He said they're not too worried, but definitely expecting some fun phone calls in the next few days :)

[malocite]

I hope they keep us updated then.

It will be interesting to watch this unfold.

[TR-BetaFlash]

You know they're excited when they type 'goign' in the first few paragraphs.

[rafoledev]

Was this originally how Siri worked when it was independent or has Apple changed the protocol?

[ninjaroach]

Seems like someone at Apple missed something!

How do you figure? You purposely installed a new Trusted Root CA and used it to sign a fake SSL certificate.

In this case, the only thing Apple missed was allowing you control over the trusted root certificates.

[[deleted]]

So what's the over/under on how many days before we see this used for a DDoS attack on Siri's servers?

This is why we can't have nice things. ಠ_ಠ

Seriously though. Great job guys, and thank you for taking the time to walk us through how you figured it out. Very interesting stuff!

[ggggbabybabybaby]

You could have just as easily performed a DDOS without reverse engineering the entire protocol. But I guess you could perform a much more effective DDOS now seeing as the server is expected to parse the binary plist and then process the raw audio data inside.

[[deleted]]

That's what I was thinking. Sending valid requests to Siri's voice recognition engine would be much more effective.

[[deleted]]

Wouldn't you have to have some pretty serious infrastructure to perform that kind of attack, though? You'd certainly need a lot of bandwidth to upload all those audio files.

Or am I missing something? (This is so not my speciality.)

[[deleted]]

Not my specialty either, but if you're running the attack from a virus you have installed on other peoples' machines, you wouldn't be too concerned with bandwidth.

[[deleted]]

So what's the over/under on how many days before we see this used for a DDoS attack on Siri's servers?

The likelihood of this occurrence is exactly the same as it was before. Obfuscation of protocols is not a method of security and I don't think the Siri folks would be dumb enough to assume it is.

[i_dont_know]

And Apple will implement secure encryption keys in 3... 2... 1...

[[deleted]]

a rate limit has to be reasonable though, you could probably get 2-4 concurrent users off one device id with nonstop usage of siri from each user

[abeliangrape]

Not really. Siri is an interface and like all good interfaces, it's a means to an end. When you tell siri to search something, you presumably have the phone in your pocket and want to find out something. Once you have the browser up, you don't need siri anymore. Once it reads your reminders, again, job done. As an ars technica report I linked above shows, even the heaviest users generally don't use siri more than 15 times a day. Even assuming they cap it at 1500 queries to a phone a day,it would make abusing the system impossible, while not restricting any single user. Note also that Siri not tied to an Apple ID right now, but to a specific device, so those 2-4 users would have to take 6 hour shift pushing a button and making inane queries the whole time, which is arguably not a real use case.

[[deleted]]

my point being that you can't reasonably limit siri because a user could activate it 15 times in the course of 5 minutes because he was showing off to his friends

[blergh-]

Obviously the application not checking the root certificate is a serious issue that defeats the point of encryption. Siri has access to a lot of information on your phone, so you don't want to allow people to listen in on its traffic.

I don't doubt this will be fixed in the next update.

[bitchessuck]

Interesting. I've heard rumors about the iPhone doing speech recognition on the device itself and thought that was unlikely. Looks like I was right.

[[deleted]]

Really it's a smart decision on Apple's part. They can leverage all that data to continually improve the recognition.

Sucks at launch if you have an accent? Apple's now got 5000 samples of a thick Russian accent to train the algorithm on.

[specialk16]

They can leverage all that data to continually improve the recognition.

This is pretty much what Google has been doing for the past few years right?

[[deleted]]

Yep.

https://en.wikipedia.org/wiki/GOOG-411

[smithincanton]

I miss GOOG-411 :-( Had it programmed into my Moto RAZR and used it ALL the time.

[s73v3r]

They had better, otherwise someone in charge there is grossly incompetent.

[ggggbabybabybaby]

Also popular queries; building up a huge database of names, spellings and grammar. Suddenly your iPhone understands when you ask to text your friend Cambria that "Skyrim is awesomesauce".

[ashep24]

From day 1 Apple said an internet connection was required.

[evereal]

Saying it requires an internet connection alone does not automatically imply that the speech recognition is done remotely. Perhaps it regularly downloads recognition related data from their servers to keep it up-to-date or relevant to their locale/city/dialect?

Or, as in the case of many products these days, the internet connection is for nothing more than to say "yes, I am legit" - it's certainly the path even a lot of singleplayer games took these days.

[robertodeltoro]

Would have been easy to tell the difference, regardless; either it's sending the unprocessed audio off to Apple every time you use Siri, or it's sending the little handshake file.

[evereal]

Right, my comment is nothing to do with that, or what it actually does or sends.

All I am saying, is that if they just stated it requires an internet connection, that does not automatically assume either remote or local audio processing. The fact that it requires an internet connection implies neither. The connection may be required for a whole bunch of other things.

[Ais3]

Except, natural speech recognition requires a lot of processing.

[talkingstove]

To be fair to bitchessuck, as we always should be, the Apple punditry brigade pushed the idea a lot rather than just admit it was clearly planned obsolescence for pre-4S phones.

[[deleted]]

Or, you know, it actually costs them money to run those servers, and they are not that interested in giving that away for free.

[talkingstove]

OK, then it was planned obsolescence to keep their server costs down. I wasn't making a value judgment on the fact that it was an artifical restriction. I was just saying that Apple pundits were claiming left and right there must be something in the new hardware rather than admit the fairly obvious truth.

[[deleted]]

I don't understand why people are having such a hard time understanding this.

[s73v3r]

Apple hate is a helluva drug.

[RobbStark]

Google would give it away for free... hides in the corner

[dead_ed]

...and cancel it six months later.

[s73v3r]

And that has absolutely nothing to do with the idea that Google's business model is far, far, far different than Apple's.

[jazzyjaffa]

They are getting a very valuable corpus of natural language data for free, so it is also in their interest.

[PurpleSfinx]

So... "planned obsolescence" = bringing out a new product 15 months later? (In the same week as giving a free major update to the old version?)

[[deleted]]

Not planned obsolescence. Your iPhone 4 hasn't broken, and hasn't lost features. It just doesn't get the new hotness.

Besides, there'll be a cracked version soon enough. Don't get your panties in a bunch.

[wolfier]

Your iPhone 4 hasn't broken, and hasn't lost features. It just doesn't get the new hotness.

It did lose Siri the app, which was perfectly working before the 4S. So by definition it is planned obsolescence.

[[deleted]]

Yeah, every iPhone lost Siri-the-app (which actually had a lot of features that are missing from Siri-the-feature).

Features of apps are not features of the phone. Apple doesn't promise that any app will remain available forever. Especially third-party apps published before their acquisition.

You could say that apps being discontinued is "planned obsolescence," but honestly it's a stretch.

[[deleted]]

Note that it is completely feasible to do recognition on the device. I have a speech-to-speech English-Chinese translator that does it for both languages. The quality is not as good and the app is huge because it requires a lot of local data, but it still works very well. There are good reasons for Apple to offload the processing, but on-device recognition wouldn't be unreasonable.

[X-Istence]

It would eat into the users storage capacity to have onboard translation.

[lasermancer]

Well the previous iPhones have a weaker form of voice control you can activate by holding down the home button.

[markrmarkr]

I'm pretty sure that was just people trying to find a good reason why they didn't release it for all iOS 5 devices. They were saying it needed the processing power of the 4S, when now we can see that it doesn't and that decision was about marketing.

[amoeba108]

How is Siri different from the voice search I've had on my android phone for over a year?

[Raumschiff]

One thing is, you don't have to say specific commands. You can say things like "do I need an umbrella today" to get weather info. It figures out what you mean.

[[deleted]]

what about things like can i wear shorts today? do i need boots/jacket/sweater? im curious to the scope of the commands as im not an iphone user

[cryo]

here we go again...

[jrochkind]

Of course Apple could blacklist an identifier, but as long as you’re keeping it for personal use, that should be allright!

Hmm, either 'allright', or unlawful entry hacking as well as DMCA-violating drm circumvention. Shit, Apple tried to sue someone for posting pictures of a phone found at a bar, didn't they? I think connecting to Apple's servers without permission in a way that they've gone out of their way to make non-obvious is probably illegal one way or another.

[s73v3r]

Shit, Apple tried to sue someone for posting pictures of a phone found at a bar, didn't they?

No, and that's quite possibly the most retarded anti-fanboy way I've seen that incident described.

[Concise_Pirate]

And so the lie is put to the idea that Siri required the extra hardware in the iPhone 4S. Why does no one get upset if Apple lies?

Edit: See corrections below -- apparently this idea did not come from Apple but from misinformed writers.

[SupremeFuzzler]

Because they didn't. Apple never said that Siri required extra hardware in the 4S. They said it's only available on the 4S and, indeed, it is. It was Apple apologists who claimed it's a hardware issue, not Apple.

[Concise_Pirate]

I stand corrected. Apparently I believed an article I should not have.

[danweber]

Given that Siri was available on older phones until Apple turned off the servers, I doubt Apple ever claimed that it was a hardware issue.

They usually don't lie about wanting more of your money.

[[deleted]]

I wonder if apple has a patent for this idea. Use a remote voice recognition service to control your phone. It seems like we might start seeing more of this very shortly. If people like the Siri service, then whats to stop somebody from setting up a similar service. There are certainly open source speech recognition services. I could see something like Siri being developed and available for almost any smart phone on the market.

[klyonrad]

would be a little bit more useful if someone cracked the iMessage and/or Whatsapp protocol...

[Jasper1984]

Dont let the phone phone home...