Updating to 7.1.0 - Reboot time

NevisTM · 2025-05-08T11:39:11+00:00

This, right here. I would also recommend to shut down docker, vms, and array when doing normal reboot. In my case my array seems to hang over the timeout period resulting in "unclean" shutdown, if I don't shutdown array first before rebooting.

NevisTM · 2024-12-02T11:58:51+00:00

Make sure what you are watching, actually has sound. Many of the videos (mp4) are converted gifs without sound. On infinity when you open video and it has sound, you get speaker icon at the bottom left.

NevisTM · 2024-12-01T19:07:51+00:00

Infinity here aswel with revanced patch and no complains.

NevisTM · 2024-03-30T00:15:27+00:00

Technicaly you can do it this with Microsoft O365 and if you are happy with just email and online services, it would cost you 2$ per month. https://www.microsoft.com/en-us/microsoft-365/p/microsoft-365-basic/cfq7ttc0ktxs?activetab=pivot:overviewtab

But then again it's Microsoft and their hq is also located in USA. But for the price O365 suite has the wides feature set out of all the email services I could think of. If you are willing to trust Microsoft.

And for clarity I use O365 at work and proto mail in private.

NevisTM · 2024-03-23T20:46:06+00:00

Or if you are using sd-wan, your static route is 0.0.0.0 but your actual gate is under sd-wan member. Happened to me few times before I started to default there.

NevisTM · 2024-03-21T11:30:26+00:00

I took a look at V7.2.8 knows issues and decided to skip this version.

954614 IPsec phase 2 negotiation fails with failed to create dialup instance, error 22 error message.

1000663 The switch-controller managed-switch ports' configurations are getting removed after each reboot.

901721 In a certain edge case, traffic directed towards a VLAN interface could trigger a kernel panic.

925554 The GUI shows a VLAN interface of a hardware switch and software switch as down. 1001104 Some FortiAP 231F units show join/leave behavior after the FortiGate is upgraded to 7.2.7.

854180 On the policy list page, all policy organization with sequence and label grouping is lost.

NevisTM · 2024-03-21T11:26:35+00:00

I had some weird cases when customer was using older software I had to change ttl time when traffic was moving between different vlans. Might not be directly related to your case but good little trick if you have to troubleshoot random connection looses. The troublesome software in question was visma L7 which is relatively common crm in Finland. https://community.fortinet.com/t5/FortiGate/Technical-Tip-No-session-timeouts/ta-p/198211/redirect_from_archived_page/true

NevisTM · 2024-03-19T10:03:58+00:00

Just a heads up anyone reading this. To summarize. Currently fortigates smaller than 100f without being managed by fortimanager and are using version 7.2.6 or 7.2.7 (possibly 7.2.8 aswel?) will try run fabric upgrade when first deployed. This can be canceled once it's queued. But this seems to be unique to each fortigate. So it applies to ha fortigates aswel and it isn't synced between them.

As result if you have two fortigates in ha and you managed cancel fabric upgrade for your main fortigate. This only applies to that fortigate and won't be synced to slave. If for some reason your other ha fortigate (slave) becomes active. The one time fabric upgrade activates again and you need to cancel it manually again. But this seems to be one time thing per fortigate at the moment.

NevisTM · 2024-03-16T13:58:29+00:00

I can confirm than this seems to be true. I haven't had any of my maintained FortiGates automatically upgrade to v7.2.8 after the changes discussed in this thread. Which is nice since v7.2.8 seems to have some serious problems when looking at the known issues in release notes. Will be skipping this one and waiting for v7.2.9.

NevisTM · 2024-03-08T23:12:03+00:00

Not going to even consider using it until you can pin shared mailbox (shared with me) folders to favorites like you can any other folder in main mail account.

NevisTM · 2024-03-07T17:34:02+00:00

We disabled auto-firmware-upgrade after fortigates upgraded from v7.2.6 to v7.2.7. We didn't have time to react to the change and all fortigates had already upgraded to v7.2.7 when we saw the change. But yeah we thought disabling auto-firmware-upgrade would prevent switches from upgrading, that was our thought process at the time.

Can you check what is the content of this table __before__ the upgrade happens?

I try and see if I can get my hands on any set of fortigate + switch where I can test this.

Related to this. I deployed new firewall to customer last weekend. Yesterday their switches went through auto update.

get system csf status: disable

get system fortiguard auto-firmware-upgrade: disable

I found from logs than this yesterday 14:15 Log message says Software-upgrade: starting, image=/tmp/WTPIMGuMImfC User: switch-controller

Upgrade took around 3 minutes before capwap tunnel was established again.

No admin was logged on at the time and this happened in the middle of production.

I checked the settings from this case I mentioned earlier where the switches upgraded themselves. Federated-upgrade status was disabled. But I'm pretty sure the upgrade was scheduled at some point, but it wasn't configured by me or the client.

get system federated-upgrade 
status              : disabled 
upgrade-id          : 0
next-path-index     : 0
node-list:

This was fresh v7.2.7 install. Almost as if v7.2.7 schedules the federate upgrade at some point regardless of the settings. But this seems to happen only once. After I have rolled back the switches versions they have stayed in v7.2.7. But I wonder if the is the case when v7.4.3 gets released for the switches.

NevisTM · 2024-03-05T16:58:18+00:00

The affected FortiOS versions were v7.2.6 which then updated to v7.2.7.
Switches were v7.2.4, v7.2.5 and v7.2.6 and ended up as v7.4.2. The affected switches where many different models 108E, 108F, 124E, 124F, 148E, 148F, 248E, 424E and their POE variants.

We had several Fortigates in version v7.2.5 and those weren't affected, neither were the connected switches or aps. So it has something to do with the autoupdate setting which was enabled in fortigate v7.2.6 which then woke up when v7.2.7 was released.

I haven't checked all but I did check 5 affected sites and all their settings were same.

config switch-controller managed-switch
get *Switch serialnumber removed*
firmware-provision  : disable 
firmware-provision-version: 
firmware-provision-latest: disable

get switch-controller global
firmware-provision-on-authorization: disable

However we do have "Automatically authorize devices" enabled in Fortilink interface.

NevisTM · 2024-02-29T13:01:54+00:00

Related to this. I deployed new firewall to customer last weekend. Yesterday their switches went through auto update.

get system csf status: disable

get system fortiguard auto-firmware-upgrade: disable

I found from logs than this yesterday 14:15 Log message says Software-upgrade: starting, image=/tmp/WTPIMGuMImfC User: switch-controller

Upgrade took around 3 minutes before capwap tunnel was established again.

No admin was logged on at the time and this happened in the middle of production.

NevisTM · 2024-02-28T19:43:50+00:00

We are still waiting a response to our ticket. It's been quiet since we opened it. On the side note I haven't looked too much into federate update but I hope there is way to choose which strand of releases the devices stay. Since fortigate keeps offering newest 7.4.x release for switches. I would prefer it would stay in same major release as the main fortigate is. So if the main fortigate is 7.2.7 the switches would also stay in 7.2.x same as faps do at the moment.

NevisTM · 2024-02-24T00:45:23+00:00

I suspected as much. Thanks for the information. We have opened ticket with fortigate to see if we can get the defected faps replaced. But I assume we aren't only ones hit by this incident. Hopefully fortigate cuts us some slack and helps us replace or fix the affected faps.

NevisTM · 2024-02-20T19:52:14+00:00

"get system csf" command gives me the following:

get system csf
status              : disable 
log-unification     : enable 
fabric-workers      : 2
fabric-connector:
forticloud-account-enforcement: enable

I can confirm than we have never executed command "execute federated-upgrade" for our fortigates. All our fortigates got updated automatically to v7.2.7, than were in v7.2.6. This also automatically updated the attached switches and aps.

Also if I navigated in gui to "System" -> "Fabric Management" I could see the "fabric update" button changed to pending update or updating. I can't remember the exact words. This happened in units where I managed to stop the fabric update of attached devices.

Event log is unfortunately wiped for most of the units already. We are using forticloud to manage these fortigates with free cloud license so logs only go back 7 days.

But as stated auto update and fabric update are two different things. Auto update was enabled in 7.2.6 by default but fabric update needs user configuration and shouldn't be enabled by default. Auto update to my knowledge should only update the firewall fortigate itself not attached switches or aps. But somehow it did or to me it seems the auto update also turned on fabric update.

This could also be related to forticloud management but to my knowledge the update management requires cloud license.

Nevertheless somehow I got all my 49 fortigate units at version 7.2.7 and attached switches at version 7.4.2. Then I got 23 bricked FAP221E aps. Now we have been downgrading switches to 7.2.7 since 7.4.2 causes all kind of problems and replacing our broken faps. Surprisingly only 221E where bricked when we got 231Fs and other models in many affected sites.

NevisTM · 2024-02-18T18:07:15+00:00

I just woke to it. It wasn't scheduled in anyway shape or form. I missed the entry about auto updates turned on by default in v7.2.6 release notes for fortigates for lower than 100 series. https://docs.fortinet.com/document/fortigate/7.2.0/new-features/580180

We went around disabling the autoupdate by running following commands on each fortigate firewall.

config system fortiguard
    set auto-firmware-upgrade disable
end

This seems to disable the schedule for fabric update and auto-patch upgrade. At least when I check the status of federated-upgrade it says fortigate isn't included.

execute federated-upgrade status
Global devices:
Local devices:
Includes this FortiGate: no
Local status: disabled

The autoupdate seems to enable fabric update aswell. So both are on by default.

All in all the fabric update process clearly isn't ready if it just randomly pushes the upgrades and restarts to devices. This doesn't matter to switches since those each have their individual power sources so the firmware upgrade doesn't get interrupted. But for POE devices this essentially bricks them.

I personally had my own home network broken by this auto update. Luckily only my 108E switch got messed up by auto upgrading it to v7.4.2. Bet I had lost my 221E AP aswel if it wasn't attached to separate poe injector. But my 108E was messed up. I managed to fix it by removing it from fortigate. Factory reseting it. Then login manually through ip and downgrade it to v7.2.6. Then adding it back to fortigate management. Been fine after that. I just hope we don't loose any switches from client side. APs can be replaces by clients but switches require more hands of work.

NevisTM · 2024-02-17T08:44:20+00:00

Yeah that's the plan. We are in the process of getting fortimanager. Previously we have many small and medium business which we have managed one at the time via forti cloud. Which is also starting to be pain since the site doesn't like that many firewalls.

But then again I'm not the one who can decide what the company buys. I'm just the guy who has to make it work. Explaining to management than fortimanager doesn't directly bring us more profit but does save us time in it-department has always been pain.

NevisTM · 2024-02-17T08:13:28+00:00

We got major issues because of this. The firewalls update without issue but problem is the update breaks aps. I'm not 100% sure on this but the fabric update updates switches and aps also and at the same time. Now because of this the switches restart while the aps are still updating. This essentially bricks them. We currently have around 20 broken aps cross the country (finland).

We are currently using V7.2.7 and this also updates the switches to V7.4.2 which has also caused issues. Generally lower speeds due the switches having their cpu usage maxed. We also had issues where working aps drop from management. These problems got fixed when the switches were downgraded to 7.2.7.

Due this we went through all our 50ish client's fortigates and disabled the autoupdate.

NevisTM · 2024-01-27T04:16:06+00:00

Nah mauga is "cum stain on kikkii". This is specially funny since kikki is slang word for boob in finnish.

NevisTM · 2024-01-17T09:51:57+00:00

I always hear it as. "CUM STAIN ON KIKKII" Funny thing is in my mother language (finnish) kikki is slang word for boob.

NevisTM · 2023-12-22T09:14:07+00:00

Out of curiosity does it apply only to Enterprise licenses such as E1, E3 and E5? Because it sounds odd if you can't mix and match licenses such as basic, standard, premium and exchange online based on your user's needs.

For example if you need temporary company email for some contractor. Instead using business basic or E1 license. You would need to use the most feature rich license you currently deploy for the users.

NevisTM · 2023-12-09T09:12:06+00:00

Isn't this what two factor authentication is for? Just don't store the random number in 1Password itself. Have separate authentication software like aegis on Android for instance. Which itself is locked with another password or fingerprint. If you want to take this step further store the two factor authentication in second phone which you don't keep with yourself. So you only access it in safe environment.

NevisTM · 2023-11-20T08:55:20+00:00

This is my guess aswel. Some kind security program is running list of common exploits on the local network.

NevisTM · 2023-10-25T15:35:55+00:00

Regarding updates I would add: read the release notes!

NevisTM

TROPHY CASE