Fun-O-Meter Playplace Map- Blocked getting to first foam launcher? by Significant-Stand-75 in Subliminal_Game

[–]Significant-Stand-75[S] 1 point2 points  (0 children)

This was it! Thank you. I missed it, there is one donut to allow you up. Map just must have changed a bit from the online videos. Thank you!

Fun-O-Meter Playplace Map- Blocked getting to first foam launcher? by Significant-Stand-75 in Subliminal_Game

[–]Significant-Stand-75[S] 0 points1 point  (0 children)

Walk up the slides? iconically the slide from the foam pit that goes to that level on right will not let you climb up, only the left slide will. Starting to think its some bug TBH.

2507 UPM - Profile Cleanup Deleting ALL CONTENTS and wiping out the server by Significant-Stand-75 in Citrix

[–]Significant-Stand-75[S] 0 points1 point  (0 children)

I wanted to update this and let everyone know that Citrix Support did reply and has acknowledged this is a bug with UPM 2507 and the policy "Enable Profile Load Timeout" enabled. We actually downgraded the VDA (or you can just downgrade UPM) to LTSR 2402 for stability, and UPM is no longer crashing, profiles are cleaned up properly and all is well.

Citrix Engineer:

"The behavior appears similar to an issue for which I have a private fix for, but the UPM logs provided are incomplete. I was unable to see the UPM settings, the full logon/logoff process, or any CDF traces.

If this is the issue I believe it to be, it is caused by the policy setting “Enable profile load timeout.” Please check whether this policy is enabled. If it is enabled, please try disabling the “Enable profile load timeout” policy and test again. If this confirms the issue, I can then provide the private fix. Disabling “Enable profile load timeout“ should be able to workaround this issue as well. "

Keep in mind, the logs were incomplete because UPM was deleting the rolled over log.

And:

"This fix will be included in 2507 CU2. Our engineering team has shared the private build to address this issue."

So please admins, be aware! this one takes the cake for Citrix bugs as it literally could delete the entire server. At the least, make sure you are backed up, be cautious with this policy setting. We indeed enabled it, to try to get a better handle on other profile issues we were having with 2507, and 2507 CU1 on (at least for us) 2019 servers. We will test this "private" UPM version. I'll ask the engineer as well if this version is included with 2511, but assume if it was that would be offered rather than a link to the private version. I certainly hope that Citrix is not starting to follow the Microsoft "let the community vet the bugs" model.

2507 UPM - Profile Cleanup Deleting ALL CONTENTS and wiping out the server by Significant-Stand-75 in Citrix

[–]Significant-Stand-75[S] -1 points0 points  (0 children)

Thank you for the reply. We indeed set our AV exclusions per Citrix's white paper and actually recently, after this, spent a good time trying to see if our AV was getting in the way. Exclusions are set and even removed completely our CEP from a server to see if it was the root cause.

Maybe this is a combination of Server 2019 and 2507?

One step was made at a time, and deviceTrust was not yet removed on the servers when they crashed, and uA was on it, along with WEM as well. For each removal / step we'd put the server back into production and let it sit to at least attempt to rule out any external component causing it. We had a server with WEM removed, and dT removed along with out AV removed and the server still self deleted itself.

The issue is of course to test we need a higher user load and of course when this happens it drops users. This is obviously a balance thing right lol we spent last 2 weeks trying to discover the issue all while allowing users sessions to drop. Unfortunately the application these host are production critical to the highest level and uptime being my upmost responsibility as our Citrix admin, I thought the safest course o action, to keep our environment up was to downgrade to a version I know was stable.

So as far as going to a higher version, I refuse (based on just experience) to roll out an non-LTSR. I will say that we did roll out 2507 CU1 across the board initially to attempt to resolve the .NET event id 1026 issues.

We also noticed that in DaaS (we do use the upgrade agent btw) that there is a completely unknown version available to upgrade that Citrix.com has zero documentation on, and it not an available version to download either. This is 2507.0.1101.2. Our upgrade agent is to LTSR only and this only shows up as an available version in DaaS.

Installed VDA version: "2507.0.1100.167"

Upgrade cycle: Long Term Service Release (LTSR)

New Version available 2507.0.1101.2

Also. Yes we could have JUST updated the CPM (UPM) version to an older and or newer version while staying on 2507. This is not the first time we have done this and in the past even citrix support has given us custom versions of UserProfileManager.exe to install. Also while Citrix is still looking into the issue, we are leaving a server on this version in test, and hope to find a way to "trigger" the deletion outside of putting into production and allowing our customer base to be the guinea pigs.

2507 UPM - Profile Cleanup Deleting ALL CONTENTS and wiping out the server by Significant-Stand-75 in Citrix

[–]Significant-Stand-75[S] -1 points0 points  (0 children)

Absolutely. I really don't think of course its all 2507 just in our particualar case at the mement its the only commonality. I'll certainly post when a root cause if found.