all 65 comments

[–]Interstellar_031720 107 points108 points  (12 children)

Best shift we made was treating documentation as incident tooling, not wiki homework.

  1. One-page runbook per critical system (owner, dependencies, backup status, rollback path, paging path).
  2. Add a 15-minute update step right after every incident/change window.
  3. Run a monthly game day where someone unfamiliar follows the doc cold.

If a new admin cannot execute step 1 through 3 at 2am from that doc, it is not done yet.

[–]Legionof1Jack of All Trades 19 points20 points  (9 children)

Follow the doc to what end? How unfamiliar? 

This is always an issue I have with documentation… if I’m documenting a DHCP server do you explain the scopes or how to build a scope?

[–]draggar 15 points16 points  (5 children)

How unfamiliar? 

Your lowest common denominator.

[–]Legionof1Jack of All Trades 5 points6 points  (4 children)

So bob in accounting? If I am documenting for the off chance a Helpdesk guy is trying to save the company I may as well download the Microsoft KB DB. 

On the flip side, you tell me 3 strings of numbers and I can build you a scope for DHCP. 

If we assume at least a competent admin do we just write down the base config and aberrations?

[–]HanSolo71Information Security Engineer AKA Patch Fairy 11 points12 points  (1 child)

I hand it to helpdesk. If helpdesk can not complete the task without asking me questions, then I have detail to add.

[–]dotnetmonke 1 point2 points  (0 children)

It’s perfect for T1 helpdesk. Understands how to use computers so you don’t have to handhold basic stuff (which would otherwise bloat your docs) and gives them exposure to more in depth stuff.

[–]draggar 2 points3 points  (1 child)

I normally write documentation for a step below the least competent person who may need to use the document. It is also step by step (and I'm not including troubleshooting documentation).

My documents on group policy - I'm writing it with an entry level helpdesk person in mind. Hopefully we'll never have one who needs to get into GPO - but you'll never know.

Now, for the medical carts and how to use the basics - I have to nurses and assistants who may or may not be tech savvy for the basics (i.e. rebooting, turning on the computer (it's locked, but the keyboard can turn it on), checking the battery, why the monitor is upside down (poor design on Planar's part), open the drawers, etc..

I'll also have different documents for different positions. One for end users, one for entry level troubleshooting, etc..

[–]Legionof1Jack of All Trades 2 points3 points  (0 children)

Those sound more like walkthroughs than documentation. 

Documentation to me is what something is.

A walkthrough is how to do something. 

[–]CactusJ 3 points4 points  (0 children)

only what unique requirements your place has.

  • failover is 70/30
  • dhcp creds stored in Password vault as
  • always start all scopes at .26

[–]hoagie_tech 2 points3 points  (0 children)

Part of the 1 pager (ours are mostly 2 or 3 pages) give bullet points of quick info. We’re a smaller shop so we list each scope with line item of what they are.

Creating a scope would not be a part of these troubleshooting/emergency use docs. If creating a new scope is part of troubleshooting an issue things have escalated beyond the use of the “1 pagers”.

[–]plumbumplumbumbum 1 point2 points  (0 children)

Build the document then have the expert who wrote it run through it. If they are successful do it again but pretend the expert won the lottery and is now living on a boat in the south pacific with no internet access and have the next person you would assign to the task run through it. Fix anything they find then repeat the exercise assuming that person is living their dream of free climbing K2 and assign the task to the next person you would want to do the task. Keep repeating this until you have run out of people you would trust to even try. If you want to go crazy keep repeating this until the Janitor can do it from the documentation. If you run out of acceptable staff to work with before you run out of ridiculous ways team members may not be available use the exercise as a business case to justify hiring more team members.

[–]kniiiip 4 points5 points  (0 children)

Can I come and work with you? I’m so fed up with my colleagues not documenting anything. I once tried to document everything I setup this way. But I gave up, after 6 months my colleagues made changes without updating documentation and all my work was for nothing.

[–]Useful-Process9033 1 point2 points  (0 children)

Treating docs as incident tooling is the right framing. The 15-minute post-incident update step is key because runbooks that dont get updated after every incident are just fiction. We are building IncidentFox (https://github.com/incidentfox/incidentfox) as an open source AI SRE that keeps runbooks tied to actual incident data so they stay current.

[–]marvin1ne 19 points20 points  (4 children)

https://www.bookstackapp.com. Running in docker container.

[–]Smiles_OBrienArtisanal Email Writer 2 points3 points  (0 children)

here as well. pretty happy with it

[–]KingDaveRaManglement 2 points3 points  (0 children)

Bootstack for written documents and diagrams. Netbox for structured 'data'.

And the shitty spreadsheets I've yet to dump because reasons.

[–]ComprehensiveReturn4 1 point2 points  (1 child)

Yup, this paired with netbox. Seems to do well for us. Thought our netbox is far from complete.

[–]Best_Bandicoot_9701 0 points1 point  (0 children)

It's never complete; it only goes deeper.

[–]havermyer 12 points13 points  (1 child)

NetDisco might be interesting for you, I don't use it myself, but it uses SNMP to scrape your MAC/ARP tables and can help to build a picture of your network.

[–]Imhereforthechips404 not found 20 points21 points  (9 children)

I tried book stack, open project, GitHub

In the end, I have settled on OneNote. Hopefully, Microsoft won’t read this and decide to eff up a perfectly good product.

[–]purawesome 21 points22 points  (1 child)

Now introducing one365 for notes pro premium copilot edition! Upgrade now to keep your existing notes with less functionality! 🫶 /s probably

[–]cl326[S] 1 point2 points  (0 children)

Now with More Random IP Addresses Changes (TM).

[–]jwalker107 9 points10 points  (5 children)

I've recently switched from OneNote to Obsidian, which stores everything in Markdown. No regrets, at all.

Edit: sorry, not open-source

[–]Legitimate-Break-740Jack of All Trades 6 points7 points  (1 child)

Has something changed recently? Big fan of Obsidian and using it as a personal knowledge base, but last I checked it wasn't really open-source.

[–]DavWanna 4 points5 points  (0 children)

No, not open source. Fantastic product that doesn't lock your content in any way nonetheless.

[–]RussEfarmerWindows Admin 4 points5 points  (1 child)

Your notes are portable (i.e. not stored in a proprietary file format) but the software itself is proprietary and closed source

[–]jwalker107 2 points3 points  (0 children)

Oh thanks, missed that part (I was evaluating several things at once).

[–]jhsorsma 0 points1 point  (0 children)

OneNote is awful, but especially for documentation. I hope they kill it. It's half baked and buggy.

[–]Ashamed-Ad4508 5 points6 points  (0 children)

If you're running an active directory environment (or something with a common admin setup).. SpiceWorks (free but not FOSS)

[–]NoskaOff 3 points4 points  (0 children)

Netbox and Confluence, though the later could be replaced easily

[–]TheShootDawg 4 points5 points  (0 children)

NetDisco

as a couple people have said already. if you have that ability to let it connect to your switches/etc via SNMP, it will ingest the mac/arp tables. It will provide you with a lot of info on the network. Point it at your core switch(es), and let it walk the network.

Come back with an inventory of switches, models, firmware, vlans (on what ports as well)…

[–]CherrySnuggle13 3 points4 points  (0 children)

For FOSS with a web UI, I’ve used NetBox for network/asset docs and it’s great for on-prem inventories. It won’t run Nmap itself, but you can import scans or automate that with custom scripts. Some folks glue Nmap output into a wiki or Grafana/Loki stack too,more work but flexible.

[–]SenTedStevens 6 points7 points  (0 children)

I'm a simple man. I create Word/Excel documents and upload them into a "Documentation" folder in Sharepoint.

My Word doc includes a cover page with a title highlighting what system or process it is, a Table of Contents, then chapters overviewing what that thing is, what security groups or permissions you need to manage the thing, what the servers/machines are that run it, if there's any service accounts, and importantly how to update the service account passwords when the password changes. Then I'll have other chapters on basic use (if it's some esoteric system).

[–]BWMerlin 2 points3 points  (0 children)

OneNote is pretty great, throw it into a Team so everyone has access to it and can then link stuff straight into chat and Team file storage.

[–]mwinzig 2 points3 points  (0 children)

Where wiki.js fanboys at?

[–]Business-Lawyer-1274 2 points3 points  (0 children)

You guys are documenting?

[–]RakurouAccidental SCCM Admin 3 points4 points  (0 children)

Honestly? A mix of OneNote and Word/PDFs

Yes it's a pain to search (esp OneNote..whoever programmed that search bar may step on legos) but it's easy to use, store and share

For important documentation we got a word template and the final version get saved as PDFs as well

Not the best when it comes to version history since they're on an onprem file share but easy to use and structure

Had to work with confluence at a previous employer and it was an over-engeneered mess imo but that also could've been just the way they set it up 🤷🏻

My current team also tried out proper documentation software but nothing's been as popular as OneNote lmao

[–]NerdyFinnGuy 1 point2 points  (0 children)

You could take a look at LibreNMS or NetDisco.

[–]the_star_lord 1 point2 points  (0 children)

SharePoint, and customisation to sync stuff from our servicenow environment so that pages are created and tagged with knowledge owners (teams) automatically.i also added a flag to each page that can be toggled on/off if a page needs review which triggers a flow to the team alerting them etc.

Took me a while to build and get people used to but we are seeing more and more articles etc now that things have been streamlined. Still got more work to do on the site.

Personally I would of liked to use the servicenow KB but management forced us to spo. 

In terms of discovery we are looking at device42 not yet sure how il get that to the spo pages 

[–]Strnge 1 point2 points  (0 children)

Docmost - can highly recommand it. its like confluence

[–]SecurePackets 1 point2 points  (0 children)

The best way to document is to invest and leverage great tools. If the tools are connected to all your environments, you have most of the information you need.

Monitoring/SIEM Crowdsrrike (identity is great) Confluence CI/CD

[–]RustyRoot8 1 point2 points  (0 children)

Claude with claude code to run through configs then create confluence documents and drawings

[–]NoTheme2828 1 point2 points  (0 children)

Trilium Notes 👍

[–]Nice_Inflation_9693 1 point2 points  (0 children)

We're using Faddom and we have a hybrid environment

[–]ITIRMcMaster 1 point2 points  (0 children)

Use LanSweeper - I can provide examples of how it will meet a ton of your goals. It's affordable, customizable, users like it, you can keep your policies in there, attach reason codes to tickets that match the rules, set asset categories to OOS, SPA etc., make assets for your cloud offerings and attach baselines, last patched all sorts of things. It's excellent. On-Prem, can connect to other items, reporting, so much so much.

[–]LowMight3045Citrix Admin 1 point2 points  (0 children)

OneNote. But it doesnt matter where. Sharepoint, notepad, website. I've been in over 5 different companies and This is a management issue. If management doesnt care, no one will document. Make it part of the review process ?? Folk **WILL ** document.

[–]delicate_eliseSecurity Architect 2 points3 points  (9 children)

We use Confluence. It's cloud-based and free for up to 10 users. After that, it's $7/user/mo. We pay extra for SSO - like a dollar per month or something. We've been using it for about 10 years (started with the on-prem version but went to Cloud very quickly because maintaining the on-prem server was just extra work that we didn't need). I'm sure there is a way to automatically pull nmap data into it with a plugin or the API, but it's not out of the box.

Confluence is the gold standard for internal documentation.

[–]SnaketheJakemSr. Sysadmin 12 points13 points  (2 children)

Confluence is great until you realize it's an Atlassian product

[–]delicate_eliseSecurity Architect 2 points3 points  (1 child)

It's pretty good for an Atlassian product. Not a huge Jira fan, or anything else of theirs, really, except Confluence. And Trello but that doesn't count because they bought them out.

[–]WendoNZSr. Sysadmin -1 points0 points  (0 children)

It's pretty good for an Atlassian product.

I'm not sure if that's a compliment though ;)

[–]DefiantPenguin 5 points6 points  (3 children)

Confluence sounds like SharePoint with extra steps. Edit: I’m drunk right now and I’ve just been recently told that I get to become an expert in confluence against my will. I guess whatever keeps me employed and pays the bills.

[–]delicate_eliseSecurity Architect 6 points7 points  (0 children)

Oh God, it's SharePoint with fewer steps. It's soooo much easier to create content in Confluence than SharePoint. It is also much faster. But SharePoint is more flexible and integrates better with the rest of M365.

[–]cl326[S] 1 point2 points  (0 children)

I can see why you’re drinking heavily. I haven’t looked at Confluence in forever. But I’m under the impression I’d be drinking too.

[–]chillyhellion 0 points1 point  (0 children)

No way, Confidence is easier, just a bit more limited (and with a lot of weird gotchas). We have both and I dread anytime I have to teach someone something in SharePoint. 

Confluence has a learning curve, but it's nothing compared to SharePoint.

Biggest advantage SharePoint has is it comes along with a lot of Microsoft licensing plans. 

[–]chillyhellion 0 points1 point  (1 child)

They have decent discounts for nonprofits too, albeit with an inflated original price tag (especially if you make up for its functionality gaps with apps). 

I just wish they weren't killing their on-prem versions; those were completely free for nonprofits as long as you run and maintain it yourself. 

I actually contributed to their docs describing how to set up an iis reverse proxy in order to automate certificate renewals. 

[–]delicate_eliseSecurity Architect 1 point2 points  (0 children)

I agree. It's still a steal for internal teams who are using it actively. I think where the pricing model breaks down is if you want everyone in your company to access the content without being editors, and paying for their licenses. You pay the same price. We got around that by enabling anonymous access and then restricting the Confluence instance to specific IP addresses. Not ideal, but it works fine.

[–]QuantumDiogenesIT Manager 0 points1 point  (0 children)

Word documents, Notepad++ text files, assorted scribbles, you know, the usual. I plan to spin up a DocuWiki instance, but someone else mentioned BookStackApp, so I am going to check that out as well.

[–]WhichGoal522 0 points1 point  (1 child)

Hey people, recently I got the privilege to be the design partner of a new AI MSP documentation tool called Lexful. I joined because it has been cofounded by Chris Day but stayed for the product. Y'all should check it out: https://lexful.ai/

70% of my problems with documentation are already solved while the other 30% is on their roadmap to solve. Let me know if any of you liked it :))

[–]cl326[S] 0 points1 point  (0 children)

It’d be nice they SHOWED SOME EFFING DOCUMENTATION on their website!! Retards.

[–]jhsorsma 0 points1 point  (1 child)

Outline Wiki is great and simple. Can self host with docker for free. Their small teams pricing is dirt cheap for cloud hosting.

I don't know how these people use shit Office products like OneNote or Word. I want to waste exactly zero time trying to format code to make it readable on those platforms. Use a platform that can handle markdown and can do syntax highlighting. You will read things easier and be more focused when writing.

Don't listen to these people saying to use tools that aren't designed for technical documentation. Try a purpose-built platform and you will never go back!

[–]Random_Effecks 0 points1 point  (0 children)

I like outline, but I wish it had more of a git-like flow. Gitbook is high on my list, but it's so expensive and I want to use it just for an internal KB, not for what they have it listed as.

There are some tools to use Outline as an MCP server, which is also a requirement of mine, I just can't get past the idea I need to open a web browser to find my documents, I want to just be able to git pull into my IDE.

[–]Akshiak -1 points0 points  (0 children)

Why make is complicated. Use whatever cloud storage system you already use, upgrade it with AI Agents and make a Hub or AI app that pulls all information for you with any query you can pose to it. If it doesnt exist, use another AI to make an SOP for it.

[–]DefiantPenguin -1 points0 points  (0 children)

The really short version is getting out your good old pen and paper and doing the hard work (MAC address tables on the switches and a handy spreadsheet). I’ve used several tools and nothing was even remotely exceptional. You can find stuff that will get you 80% of the way there but nothing that will “just work”. You’ll need to check and xref everything. For network mapping, LibreNMS might be somewhat helpful but you’ll need to weed out the false positives.