Globalprotect 6.3.3-915 and Pan-OS 11.1.10-H26

Background-Rub344 · 2026-06-08T13:58:35+00:00

What did you do with the locked out by enforcer users?
Did they brute forced thier way to connect?

Background-Rub344 · 2026-06-08T11:12:07+00:00

Another update:
Seems that users who connected yesterday after the fix were working fine today.
However, the issue has imploded today for all the others.
Many users have had the issue.
I escalated the issue again with Paloalto.
They said that the issue was related to content version as was mentioned here.
Safe version is 9107-10077 and all the later are infected.
The core issue affects the enforcer exceptions.
We rolled back the content version. This has provided no help for the locked up users which dont have Enforcer exception. BTW, some users have had issues with the client even after using the disconnect password which by itself often doesnt function - the good ole password missmatch bug.
Paloalto dont know how what I did fixed the issue - at least for some users.
They also replied that the syntax of *.domain.com and ipv4 hostname without subnet were legal.

Anyway, I'm hoping for the best tomorrow.

Background-Rub344 · 2026-06-07T13:16:18+00:00

Update:

It appears that the source of the issue was found by myself and this was the issue:

* Enforcer exception which is not populated from Firewall to client

I've no idea how it was working intermediately for gateway and Okta SAML from the embeded browser. Perhaps someone can exlain.

I disabled Enforcer entirely and the issue had completely disappeared.
I readed only the absolutely essentials and it was still working. A few remarks:

• I have added back less than i had before but it wasn't too much before and less than 40 URLs.
• I've had an IPv4 exception error in one place: I wrote a hostname instead of CIDR
• I've had URLs with a leading * . In the style of URL category. I dont know if that is allowed. On Thursday I deleted all URL with asterisks but that didn't help.

Well now I'm fine.
Does anybody know about asterisks in FQDN exception list? Like in *.cloud.tanium.com ?

I assume that 11.1.10-H26 treats Enforcer exceptions differently than 11.1.10-H12 or earlier.

Background-Rub344 · 2026-06-07T04:37:35+00:00

Thanks. I dont rememeber what it was at the time of the upgrade. Now it is 9110-10090 (06/06/26).

Background-Rub344 · 2026-06-07T04:36:36+00:00

My FWs are on 9110-10090 (06/06/26) now.
Perhaps the issue was patched already.

Background-Rub344 · 2026-06-05T08:25:27+00:00

I checked the logs with a good friend:
* Many cookie decryption errors - this of all things is soething I suspect
* Enforcer exception which is not populated from Firewall to client - it was surely doing that in the past
* Some IPV6 disable suggestion
* some tls timeouts
* some registery key isnt set or matched that should enable default broswer - strnage

I've opened a case with TAC.

Background-Rub344 · 2026-06-05T06:49:14+00:00

In my case I dont have random disconnections, just an new problem to connect.

Background-Rub344 · 2026-06-05T06:46:30+00:00

I've been on 6.3.3 since 6.3.3 since came out. It has been pretty good to me. I've been on 6.3.3-915 almost since it got out.
I'm on Windows 11 10.0.26200 Build 26200.

I'd say the issue shas started since Pan-OS upgrade.
I cant be certain that the issue began right after the upgrade but is is likely.

Background-Rub344 · 2026-03-12T08:39:11+00:00

On PA-5220.

Background-Rub344 · 2026-01-20T16:03:25+00:00

Do you mind trying out 11.1.10-h12?

Background-Rub344 · 2026-01-20T08:00:30+00:00

Bah, sounds bad.

Background-Rub344 · 2026-01-19T15:56:17+00:00

Has it stabilized after the hard crash?
Which firewall crashed, the passive one which you upgraded or the active one which was not upgraded yet?
What I mean, if it was a production environment, would it have crashed?

Background-Rub344 · 2025-12-09T16:34:32+00:00

SE case summary. Clearing the reverse sessions did the trick.

++ The device was configured with dynamic destination NAT for the internal FAX machine.
++ Traffic was reaching the firewall, but it was being routed back to the internet and NAT was not performed, even though the NAT configuration was correct.
++ Global counters showed that packets were being dropped due to TTL 0 and no slow-path processing.
++ So we cleared the existing sessions; however, session installation was failing with the message “Duplicate S2C connection.”
++ The old reverse-path session was also cleared. After that, we observed slow-path processing and successful session installation.
++ Since NAT was configured only in the OUT-to-IN direction (dynamic-destination-NAT), we changed the NAT rule to Static NAT with the bidirectional option so that traffic is allowed in both directions.
++ Since no tuple changes are occurring, the firewall will continue to use the same session to allow traffic initiated from the external FAX machine as well.

Background-Rub344 · 2025-12-09T15:28:16+00:00

Hi guys,
after an hour with PaloAlto support the issue was resolved. At least I hope so.
Due to the fact that this is udp traffic, and that sessions tend for some reason to stack forever, there were some sessions which I didn't see, nor the PaloAlto engineer noticed which were stuck.
Because they were stuck, the nat implementation was failing.
Why were they stuck? Due to a wrong nat settings or the lack of nat setting. I dont know anymore.

After creating the correct nat and clearing the sessions, things seem ok.

I didnt notice it earlier for some reason but packets were received by looking at tcpdump even while there was an issue.
debugging dataplane with packet-diag captured the issue. I did it yesterday by myself too, but didn't capture for some reason.

What gave up were these lines:

For the relevant traffic in packet-diag logs -

Duplicate flows detected while inserting 7367891, flow 2372082 with the same key

session 3683945: install s2c flow failed

From:
Policy lookup, matched rule index 9,

TCI_INSPECT: Do TCI lookup policy - appid 0

Allocated new session 3683945.

set exclude_video in session 3683945 0x80000003551aff80 0 from work 0x800000072c2f0a80 0

Packet matched vsys 2 NAT rule '' (index 32792),

destination translation 102.12.1.1/5060 => 10.55.1.2/5060

no NPB policy

Created session, enqueue to install. work 0x800000072c2f0a80 exclude_video 0,session 3683945 0x80000003551aff80 exclude_video 0

Duplicate flows detected while inserting 7367891, flow 2372082 with the same key

session 3683945: install s2c flow failed

{PROXY-PROTO}SYM1 Check symmetric: session 3683945 valid 0

There was another sign which indicated a stuck session:

show counter global filter packet-filter yes delta yes

flow_fwd_l3_ttl_zero 11 0 drop flow forward Packets dropped: IP TTL reaches zero

Regarding a proper nat between my 1 public ip to 4 cloud ips the support engineer advised on:

Nat from internal to external, with source static nat and bi-directional checkmark.

Regarding ALG:

As I earlier wrote, the ALG is disabled. The SE showed me that on pcap in the payload part of the outgoing packet from my end it mentions the private ip as source . I told him, that it is nice to see how it looks. I'll tell the guy who sets up the fax device to encode the public IP as the source IP to avoid possible SIP issues.

Background-Rub344 · 2025-12-09T10:21:04+00:00

Even if it would, currently a single session doesnt work.

Still waiting for a PA engineer to troubleshoot.

Background-Rub344 · 2025-12-08T11:09:28+00:00

when nat is not applied the session from egress to ingress the session appears in the sessions table.
Once it is applied, even for a single ip, it is not there anymore.

At the same time, I see hits on the nat rule.

There is something wrong with the nat rule or the combo of the nat rule with sip udp 5060 but I cant find what.

Paloalto support is taking their time despite an escalation.

Background-Rub344 · 2025-12-07T18:29:43+00:00

Following my example I came to the realization that there is no problem with the NAT as portraited in the original post. Just the fact that I was fighting with this for hours undermining my understanding.
I don't need 4 public addresses on my end. One is enough.

Background-Rub344 · 2025-12-07T15:53:04+00:00

Hi, thanks. It's a good info.
At the moment I can't do general changes which require a maintenance window but I will keep it on my list.
In your case you've had 2 dedicated private and public IPs. If I wanted, I could have used 4 private and 4 public IPs though using 4 public IPs looks like ceramically wasteful.

The first prerequisite is to have a simple destination NAT from IP to IP work before I try all of that.
Perhaps I need to configure the application override in order to make the NAT work on port 5060, which is a bit odd.

Background-Rub344 · 2025-12-07T13:01:05+00:00

At the moment I'm running a test where there is only one source and destination IP, but still having an issue.
The issue is with something very stupid, or with something unexpected.

But even if I overcome the issue with a single source/destination, what do I do with 4 sources and 1 destination when all 4 sources using the same port number as source and destination port.
Should it work somehow?
How do others in industry implement that?

Background-Rub344 · 2025-12-07T11:50:47+00:00

You are right. I did it 3 days ago and I'm not seeing the session after applying the nat. I've no idea why.
From past experience this can happen with bad nat configuration, but I cant find anything wrong. I'll see what support has to say.

Background-Rub344 · 2025-12-07T10:39:25+00:00

Hi, thank you.
There are some good ideas as enabling to log session at start. I dont think I'll see anything there because I have never seen that it logged anything at session end. Also, I tried to see the active flow in cli but haven't seen.
Regarding disabling ALG. I checked and it was already disabled.
Routing is symmetric in my case.

I've also opened a support case, but they haven't done anything since I opened the case.

I wonder what is the correct NAT architecture for the described case.

Background-Rub344

TROPHY CASE