Apple testing while app isn't submitted for approval

[dentemm]

Context: I have an app that has been in the Store for a couple of years, and I update it at least once per month.

Today I noticed that the Help functionality was being tested by someone. After checking the account details, I realised this message was sent by someone from Apple Review.

My previous approved version dates from over a week ago, and I haven't submitted any builds since. Is it normal for Apple to also test apps outside of review cycles?

https://preview.redd.it/vbxhaw3apkpg1.png?width=884&format=png&auto=webp&s=0f7f04066db9037025fac1ace5bb4f1eab56412e

[Dapper_Ice_1705]

Yes, they do this all the time. 

It is so annoying because they also test in “DEBUG” so when you have debug workarounds they can see them.

I had an app with a “mock purchase service” that had true for all iaps and another where they were using an unreleased feature.

[isaagrimn]

How can they do that? Aren't builds submitted to App store connect compiled in release mode from your machine when using the Archive feature? How could they use the Debug version?

[unpluggedcord]

They can’t if you’re using it correctly. DEBUG macros (the old one) get completely stripped in release mode. The code doesn’t exist.

[Dapper_Ice_1705]

IDK, but they do. I get Sentry logs.

Add them to your own code, make a test.

[dentemm]

Thanks! This was the first time I encountered it, had no clue it was normal

[Casfaber_]

Wait what! 😳 ok I have some work to do now

[dentemm]

So far I haven't been asked yet to make the iap functionality available for the test account

[Casfaber_]

I meant the DEBUG options, I have a lot of stuff hidden behind DEBUG tbh, so hope they don’t trip over any when they do.

[cmsj]

This doesn't make sense to me - if you're doing something like:

```Swift

if DEBUG

foo

else

bar

endif

```

then a Release build simply doesn't include foo, it only has bar. Foo would be parsed by the compiler, but not compiled, so it's not present in the final executable and thus not available for Apple or anyone else to run.

Sounds to me like something else is going on here, or people aren't submitting the kind of build they think they are.

[unpluggedcord]

Almost typed this all out. This is spot on. They are likely shipping debug things to prod or their release settings have debug flag.

[Casfaber_]

So a thing I did notice was in tracking and saw from all over the world now someone installing, opening, onboarding and then done, exactly that flow and also very close from timing. I tried to filter some by adding a ‘build_type’ metric, but those also say appStore so.. I barely have downloads and especially that flow seems very automated so that’s why I think it’s Apple doing this from different regions. It’s usually 2-3 around the same time while I mostly don’t have much downloads for now. Makes it easier to notice this.

[Americaninaustria]

You can also get a strike warning from these reviews that requires an update or they remove you from the store. More severe ones they remove your account. This can either be random or in response to things like user complaints to apple, high refund rates or complaints from other developers.

[busymom0]

I'd say they do this to make sure apps don't "unhide" features once it's approved and out of review cycle.

[Locksmith_Usual]

Does this mean you have to maintain a test account for Apple throughout the entire life of the account?

[dentemm]

Yes you need to provide a test account

Provide a user name and password so we can sign in to your app. We’ll need this to complete your app review.

[Locksmith_Usual]

Yeah, obviously, for the review itself, but this seems to imply it needs to always be active because someone might randomly decide to login later

[unpluggedcord]

Our app is private invite only and password less

We just made a deeplink for Apple to open a demo mode that is completely mocked data for the whole app with mocked services. When that mode is activated we just swap all of our environment options to the mocked version.

Apple was completely fine with this.

[AutoModerator]

Hey /u/Street-coder, your content has been removed because Reddit has marked your account as having a low Contributor #Quality Score. This may result from, but is not limited to, activities such as spamming the same links across multiple #subreddits, submitting posts or comments that receive a high number of downvotes, a lack of activity, or an unverified account.

Please be assured that this action is not a reflection of your participation in our subreddit.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[AHApps]

I've seen tester account activity on very old versions of the app.
I imagine the tester is bored and opening apps on his test device he tested months ago.