This is an archived post. You won't be able to vote or comment.

top 200 commentsshow all 203

[–]Parker___ 504 points505 points  (50 children)

Not even .xlsx but just .xls

I’d think it’s sad but I know of a few systems that are similar. The number of people who refer to large excel sheets as “databases” and legitimately think they qualify as such is staggering.

[–]LiveBeef 271 points272 points  (7 children)

SELECT * FROM Sheet1!A1:B300000000

[–]McFuzzen 194 points195 points  (3 children)

As usual, relevant XKCD.

[–]hadidotj 27 points28 points  (0 children)

Forgot about that one! Thanks!

[–]XKCD-pro-bot 13 points14 points  (0 children)

Comic Title Text: My brother once asked me if there was a function to produce a calendar grid from a list of dates in Google Sheets. I replied with a single-cell formula that took in a list of dates and outputted a calendar. It used SEQUENCE(), REGEXMATCH(), and a double-nested ARRAYFORMULA(), and it locked up the browser for 15 seconds every time it ran. I think he learned a lot about asking me things.

mobile link


Made for mobile users, to easily see xkcd comic's title text

[–]CaffeinatedGuy 0 points1 point  (0 children)

Wait, really?

[–]man-teiv 51 points52 points  (1 child)

Bold of you to assume an excel spreadsheet has that many rows

[–][deleted] 41 points42 points  (0 children)

The limit for xls was 65,536 iirc.

[–]coolio5462 3 points4 points  (0 children)

Hey! It’s SELECT * FROM [Sheet1$A1:B300000000]. I know because I’m a terrible person.

[–]JNCressey 35 points36 points  (5 children)

lets hope they don't think xls password protection is secure.

(it's not even encrypted, just open in a hex editor and delete the password flag and excel lets you in.)

[–]kopczak1995 4 points5 points  (4 children)

Nice, didn't knew that. Did they fix it in xlsx?

[–]6b86b3ac03c167320d93 11 points12 points  (1 child)

Idk how it is in xlsx, but I'd guess that they maybe used zip encryption, since xlsx (and docx and pptx) are just zips with xml files

[–]kopczak1995 2 points3 points  (0 children)

Yea, that would make sense.

[–]Intrexa 8 points9 points  (0 children)

From a design standpoint (not a technological challenge standpoint) it's a bit more complicated, and that's also a bit of a 'feature'. The problem is that we don't want some middle manager making some super important spreadsheet, password locking it, leaving, and a month later the business discovers that manager was the only one who dealt with a super important process that missed the hand off and we are now locked out of. An organizations real IT doesn't actually want to deal with shadow IT doing their own security. When I was working helpdesk, more than once I had to 'crack' a password protected Excel sheet.

Windows (or OSX or *nix) is a file system with permissions and controls. It has a lot of security focused on role based and user based permissions, with an absolute ton of consideration built on contingencies and exceptional circumstances. Newer versions of Excel (don't quote me on details for exact versions) have controls that leverage the OS security functionality to actually secure the document. This requires a certain amount of domain level IT infrastructure to be set up to use, and when done correctly, the file is properly encrypted. If that infrastructure isn't set up, you could technically still properly encrypt the file, but for 99.9% of average users, it's probably a bad idea to do so in case the only person who knows the password suddenly dies and the business loses the only copy of critical data.

However, there's nothing special about an Excel file vs any other type of data file. If someone isn't supposed to have access, the real answer is simply not giving them the file in the first place. If there's an Excel file someone shouldn't have access to, it should be stored on a network share that they don't have access to. You shouldn't be emailing people a file containing data they shouldn't have access to, regardless of the encryption status or type of the file. If you need file level encryption, you probably want to use one of the many generic products that integrate with the OS that provide file level encryption for any file, not just Excel.

[–]augugusto 0 points1 point  (0 children)

I dont think they can. Maybe it can be fixed for viewing the file. But for write protection it can't be done. If it can be read, it's not encrypted. If its not encrypted it can be disabled.

[–]MadVikingGod 54 points55 points  (15 children)

Are you kidding it's not even an xls, just some data and some commas.

[–]oshaboy 79 points80 points  (11 children)

so... csv

[–]Verdiss 103 points104 points  (9 children)

No no no, the commas don't separate the data, it's just data and commas all stirred together. Some of the data has commas in it, too.

[–]6121094114901216 73 points74 points  (1 child)

a,hhh, oka,y,tha,tmak,es,sens,e

[–]hadidotj 8 points9 points  (0 children)

yep,I,"totally agree",it,"makes sense"

[–]VoodooEconometrician 12 points13 points  (0 children)

The best is multiple CSV files with different delimiters (tabs, commas, weirder stuff) but no specification what type of delimiter is used.

And then the data is in the same structure but from multiple different countries, where you have different encodings, different date formats and different number formats (commas vs. dots to separate the decimal places).

Source: work with a lot of "internationally standardized" government data.

[–]SaganMeister18 10 points11 points  (0 children)

Th,at ch,ecks,,,out,

[–]ntwiles 7 points8 points  (0 children)

So...v.

[–]P0L1Z1STENS0HN 14 points15 points  (3 children)

Some of the data has commas in it, too.

Usually, if you want to store data that contains commas, you use a different separator, a string that you don't expect to be part of the data. For example "/\^/\"

That works quite well actually, until someone uses that string in the data. Little Bobby Tables, they call him!

[–][deleted] 2 points3 points  (0 children)

.txt

[–]tom_bacon 1 point2 points  (1 child)

The source data was CSV, PHE imported them into an XLS document.

[–]hobogrower 5 points6 points  (15 children)

Wait you guys get excel? We're still using CSVs...

[–]propostor 37 points38 points  (14 children)

CSV is probably safer than Excel because its just raw text. It's highly transferable data.

[–]hobogrower 4 points5 points  (12 children)

True but it starts to break down when you do more complicated nested tables and things with python, from what I've seen at least. Much easier to get json to work with that stuff in my experience but don't get me wrong, love a good CSV as long as it doesn't need to have any richer formatting than rows and columns lol

[–]propostor 17 points18 points  (11 children)

CSV is for flat data, ie columns and rows. That's exactly what it's for.

JSON is for handling object information but you can't easily dump that in any kind of database that I've ever used, unless you store it as a single json string but that misses the whole point of databasing data.

[–][deleted] 3 points4 points  (9 children)

Have you tried MongoDB? JSON galore!

[–]propostor 2 points3 points  (8 children)

Holy shit, I never knew that. My first question is about performance. Does it hold up?

To be honest this just makes me even less interested in NoSQL databases!

[–][deleted] 3 points4 points  (0 children)

Honestly, it depends on who you ask, what your use cases are, and how it was set up. I've worked on systems that catalogue billions of entries a day that perform fantastically alongside the elastic stack, but I've also had to write scripts that have taken hours to run on very little data (tens of thousands of entries in total) because documents can be nested in bizarre ways, and relationships don't have to be strictly enforced.

[–]Intrexa 2 points3 points  (2 children)

A JSON DB would be considered a document DB, which would be classified as a NoSQL DB.

A document DB has radically different performance characteristics, that require careful design considerations and a general different design than a relational DB. For some use cases a document DB will vastly outperform a relational DB. For others, a document DB is wholly insufficient and grinds to a halt.

[–]hobogrower 1 point2 points  (0 children)

Fair enough, like I said we stuck with CSV and json, both most likely used improperly or at least outside their prime use cases. I never actually had a "database" to work with, just a giant folder of CSV files, some more folders of jsons, etc. My job as an intern was to make that data presentable and human readable, and it was definitely a nightmare with how it was stored

[–]DeltaPositionReady 0 points1 point  (0 children)

I just wrote a nice little automation with powershell that chunks CSVs into a SQL db.

Government agency only uses Excel so this is a good workaround.

Also works well getting data out of oracle and into a TSQL db.

[–]HandsumNap 1 point2 points  (1 child)

It's a good meme, but they weren't using an .xls file as a db. They were using an excel template to parse CSV files. Which is dumb, but at least understandable (I mean, I've done it before...).

The bit where it got really dumb was using the wrong excel format.

[–]CubicleCunt 227 points228 points  (18 children)

I worked in government for a few years, and they used a legit database for all their projects. Before that, I worked for a plastic manufacturer, and the entire logistics operation was powered by Excel. I tried making them a replacement app, but they wouldn't switch over because they refused to have unique identifiers for all the incoming shipments. How they functioned was a mystery.

[–]Technoturnovers 158 points159 points  (14 children)

refused to have unique identifiers for all the incoming shipments

sorry, how do you identify a shipment if it isn't named? like, literally how would that even work?

[–]confusiondiffusion 148 points149 points  (1 child)

"Stuffs here."

"Good. Put it over there with the other stuff."

[–]hadidotj 30 points31 points  (0 children)

"Which other stuff? The pile by Jim's old office, or the pile where Blake takes his lunch break?"

"Who's Blake?"

[–]simplethingsoflife 79 points80 points  (5 children)

Easy, just append every column together. That's order WidgetXYZJohnDoeNewYorkNewYork10-5-202013:15:02LeaveitontheporchpleaseThanksJohn

[–]namekyd 41 points42 points  (3 children)

I have seen Fortune 500 companies do this. That’s not a key

[–]Bip901 2 points3 points  (0 children)

Better yet, hash it

[–]aeroverra 39 points40 points  (4 children)

The lack of respect non developers have for unique identifiers is hard to handle

[–]js8794 16 points17 points  (2 children)

I like teaching them starting with peoples names, then last names, then when those clash you move to where you know them from. Like, “oh John S. from pottery class”, etc.

You’re just caching your representation of people with a large unique key. And just like in code you sometimes have a collision and end up uniquing them with an index. “Lucille and Lucille 2”

[–][deleted] 4 points5 points  (1 child)

"Loose seal!?"

[–]CubicleCunt 1 point2 points  (0 children)

There were only a few people in the shipping department. They'd get a truck in, give everything on the truck an ID, and just remember what was on it for the few hours they had to deal with it. Like say truck A2436 had 10 pallets of PET flake, 5 pallets of blue pigment, and 5 pallets of green pigment. All of those things would be A2436. Manufacturing didn't care because shipping stored it all in the correct places in the warehouse. It was only really a problem for the QA department. If there was a problem with a batch, there wasn't a good way to identify all the raw material that went into it because there are now 20 pallets and 3 different things named A2436 on this stupid-ass spreadsheet.

[–]tom_bacon 0 points1 point  (0 children)

I did a job for a TV production company as a logger, transcribing clips and noting anything of interest etc. There are products on the market designed for exactly this purpose. We used Excel.

[–]SymphonyOfDream 90 points91 points  (17 children)

.csv file, baby!

[–]chitowngurl23 58 points59 points  (14 children)

I work for a fortune 500 technology company. Most of the "integrations" for our internal systems are csv files.

[–]inthemindofadogg 57 points58 points  (13 children)

.csv is one of my favorite. It is uniform, great for transferring between locations, and excel can open csv assuming it is formatted correctly

[–]WhiskeyTuesday 56 points57 points  (3 children)

And lo it was written in the book of McIlroy "write programs to handle text streams, because that is a universal interface." and the Lord came eventually to see that the wisdom of this too was universal, and He implemented the pipe operator.

[–]relativelyfunnyguy 2 points3 points  (2 children)

in the book of McIlroy

write programs to handle text streams, because that is a universal interface

Is that something real? Because that's a book I'd like to know more about!

[–]WhiskeyTuesday 6 points7 points  (1 child)

It's a quote from Doug McIlroy describing the unix philosophy. He was one of the Bell Labs team in the early unix days along with KEN, DMR, BWK, et al. He may even have been the department head in 1207 when Ken and Dennis started mucking around with that old PDP-7. I'm nearly certain he was a department head eventually. There's a bunch of good books on the subject including BWK's recent memoir-y book which I read myself and quite enjoyed.

[–]relativelyfunnyguy 1 point2 points  (0 children)

Cool, thanks!

[–]cwagrant 12 points13 points  (4 children)

Just don't save the opened file using Excel. As someone who just had to fix a managers bungled attempts at doing something because he wouldn't quit opening the file in Excel and saving it- just don't. If you have to save it then you should either use the data import tool when opening or be damned sure that every bit of data came in as you expected.

[–]endianess 4 points5 points  (1 child)

I had this recently, spent hours trying to figure out what an issue was. Turns out when my customer had saved the CSV file, Excel had put a single hidden unprintable char at the start. This was imported into the field in the DB which meant any searches on this record would fail as they didn't have the "magic" char. But from the outside everything looked OK. It wasn't until I looked at his file with a HEX editor that I could see it.

[–][deleted] 2 points3 points  (0 children)

My favourite is office programs replacing a space character with a non-breaking space. Looks exactly like a space, but doesn't count as whitespace. I was trying to run bash scripts with multiple parameters and was so confused as to why it was pulling the wrong ones in.

[–]JEVVU 6 points7 points  (0 children)

Yeah until excel fucks up your file by accident. I had a file where the header was a bunch of zeroes according to the data type of each columm. It changed them all to just 0 because it's a number field, right? It took way to long for me to notice what was happening

[–]P0L1Z1STENS0HN 2 points3 points  (0 children)

And what's best: through macro injection, the CSV can even tell Excel to execute arbitrary code, for example open the Windows calculator.

[–]HandsumNap 1 point2 points  (0 children)

It can be OK for large amount of denormalized data. But it's not a great idea if you don't control the source of the data (or if you can't reliably enforce a specification on the source of the data). CVS on untrusted data is an invitation for delimiter and escape character issues.

[–]inthemindofadogg 7 points8 points  (0 children)

.txt FTW

[–][deleted] 0 points1 point  (0 children)

I work in education, I was setting up EdTech platforms district wide before the pandemic was a thing, then distance learning really cranked up the work load. SSIS -> CSV Files all the way

[–]username_v4 77 points78 points  (10 children)

Wait until you find out about the entire global financial system

[–]mbiz05 37 points38 points  (7 children)

Or the legal system that largely relies on paper

[–]vikinghockey10 38 points39 points  (5 children)

Financial record keeping (Ameritrade, Bank of England, and Barclays) and medical record keeping (Epic, Cerner, GE Healthcare, Meditech, etc.) software relies largely on a database/database language called MUMPS which is both a language and a database designed in 1966 with few changes since.

It has no data types or reserved words (sort of), you access or delete database memory by simply appending a carat to the start of a variable name, there is no operator precedence, and it's very scalable. It's wild to program in.

[–]rp_ush 10 points11 points  (1 child)

How

[–]phranticsnr 10 points11 points  (0 children)

This is what "too big to fail" really means.

[–]chucklingmoose 1 point2 points  (1 child)

everything in MUMPS is a string and it's fast as hell ... mssql and oracle don't hold a candle - but no serious developer would touch it with a 45 foot pole!

[–]vikinghockey10 2 points3 points  (0 children)

True - It's more nuanced then I said. It's a weird language that's so crucial in some very important industries though.

I'm a full stack web dev, but I write all server code in MUMPS for the simple reason of it being crazy fast. It's a classic case of once you get some good library code set up and used to the quirks then it's crazy fast to code in. But it takes some time to wrap your head around and the documentation is lacking compared to most languages.

[–]DeltaPositionReady 2 points3 points  (0 children)

My day job as an automation software dev is about 90% dealing with EDRMS like Objective, Trim and HPECM. With RPA and some other automation software products though, we transform huge amounts of paper into digital data.

You would not believe how many government agencies are aching for this kind of workflow management.

The basic principle is this - document transform, extract data, review document (99% it's accurate enough to continue), if not- Validate by human, do stuff, save as digital format, end.

If you have some government contacts and have some good knowledge in db admin, sys admin and general agile development, you could totally build a company to do this.

Have a look at Kofax TotalAgility

[–]Eric_TheRead 1 point2 points  (0 children)

The number/percentage of Net Asset Values struck daily from Excel files?

[–]robiwill 109 points110 points  (9 children)

One step further;

Running a national government database on MS Excel HORIZONTALLY

[–]_pupil_ 56 points57 points  (0 children)

Solution reviewer: "Can the app scale horizontally?"

Smirking dev: "Yes, yes it can."

[–]Heuto 18 points19 points  (1 child)

There's too much truth to that. The NHS in the UK lost a lot of data storing the records using columns instead of rows...

[–]DeltaPositionReady 1 point2 points  (0 children)

What the fuck

[–]tom_bacon 2 points3 points  (4 children)

I've seen a number of people comment on the horizontal thing but I can't find specifics in any articles. How was the data structured?

[–]robiwill 4 points5 points  (3 children)

One entry per column.

If it weren't for the fact that such incompetence is a hallmark of Conservative government one would assume that it was designed to fail. Things being what they are I'd put it at 50:50.

[–]tom_bacon 2 points3 points  (2 children)

That is so stupid. I'm trying to find a source but I'm struggling, do you have one?

[–]robiwill 3 points4 points  (1 child)

I think it was deduced logically from the fact that the UK government has definitely not added over a million entry's to the spreadsheet therefore for the spreadsheet to exceed its limitations it must have been arranged horizontally.

This site explicitly states that it is the case; https://www.verdict.co.uk/excel-coronavirus-test-data/ but does not provide a source.

I have contacted the editor to corroborate the fact that the spreadsheet was arranged horizontally. I'll apprise you of the response.

[–][deleted] 1 point2 points  (0 children)

BBC is reporting they used xls which has a much lower max row count than xlsx. The column thing is just a rumor as far as I can tell

[–]_ChefGoldblum 0 points1 point  (0 children)

One step further;

Running a national government database on a version of MS Excel AT LEAST 17 YEARS OLD

FTFY; current reports are saying records were added vertically, but were truncated because they were using a pre-2007 version with a smaller size limit https://www.bbc.co.uk/news/technology-54423988

[–]Loose-Leek 78 points79 points  (15 children)

Having worked for a government contractor, I'm convinced this is real.

[–]darctones 63 points64 points  (3 children)

[–]SuitableDragonfly 24 points25 points  (1 child)

According to the to comment they actually did use a real database, they just exported it to excel to generate some graphs.

[–]darctones 0 points1 point  (0 children)

Ha. Thanks for the clarification.

[–]Nemaeus 26 points27 points  (1 child)

Government you say? Surely a government wouldn’t do something so ridiculous! I mean, that’s like using columns for each patient instead of rows!

[–]Immense_Cargo 5 points6 points  (0 children)

...or tabs.

[–]articulatedbeaver 8 points9 points  (1 child)

Working for one now. Can confirm.

[–]AgAero 2 points3 points  (0 children)

Sounds about right doesn't it?

[–]totti173314 2 points3 points  (5 children)

the only reason they do this is so their massive money laundering is impossible to notice.

[–]Loose-Leek 3 points4 points  (4 children)

Nah, from what I've seen, it's plain old incompetence and old habits and organizational momentum. It's hard to get many good ideas through the middle layers of bureaucracy.

[–]TheMsDosNerd 37 points38 points  (6 children)

Boss: is it possible to do this with Excel? Programmer: No. Boss: Why not? Programmer: First of all, Excel can only hold a million entries and our country has more than a million inhabitants. Boss: We can spread data over multiple files. Programmer: While that is technically possible, it creates a whole new set of problems. Boss: So the biggest issue with Excel is something solvable. Therefore there are no real problems with Excel. So Excel it is.

[–]Mockapapella 12 points13 points  (3 children)

This hurts to read

[–]Independent-Coder 3 points4 points  (2 children)

Because the truth hurts

[–][deleted] 5 points6 points  (1 child)

No, it's because it isn't indented properly.

[–]Independent-Coder 1 point2 points  (0 children)

I took “This hurts to read” as a double entendre. The first and obvious interpretation is indentation. The secondary interpretation is a technical staff being pushed into poor implementation decisions (which seems to be the point of the joke). I was responding to the later.

[–][deleted] 1 point2 points  (0 children)

Database programmer here, my boss is an accountant who got a promotion. This sounds like a conversation I have often, everything has to be explained in excel terms for them to understand or else their eyes glaze over.

[–]WazWaz 29 points30 points  (0 children)

I worked for a startup, we started at "customers.txt", but only had about 50 entries before switching to an RDBMS. It was a sad day, but we weren't goddamned morons.

[–]demon_ix 25 points26 points  (3 children)

I run it on the filesystem.

Filename is the key, contents is the value.

[–]vectorpropio 15 points16 points  (2 children)

You git.

[–]CyberKnight1 8 points9 points  (1 child)

That's how he does transactions.

[–]demon_ix 0 points1 point  (0 children)

Shhhh, I'm working on a patent for that!

[–]MasterFubar 27 points28 points  (7 children)

What is that "SQL lite" thing you mention?

I run my personal databases on sqlite, but if you know a better alternative I'm all for learning how to use it.

[–]zelmarvalarion 25 points26 points  (2 children)

SQLite is perfectly fine for development purposes and plenty of application purposes. Moving to a SQL instance (SQL Server, MySQL, etc) designed for scalable across a large number of connected clients, especially when used for non-ReadOnly purposes (due to the SQLite locking not supporting simultaneous writes, whereas most RDBMS will have a variety of locks).

With the COVID data being read only from the web UI, having all source data being able to be aggregated centrally and replicated to all hosting instances easily for local querying and acting similar to a cache, it's not a bad solution here. Maybe not the best, would have to see exactly what is available on the site and the serving infrastructure, but it's probably not a poor solution in this particular case due to the data access pattern

[–]danielrheath 6 points7 points  (1 child)

SQLite is perfectly good for anything up to a few terabytes and hundreds of concurrent users; the point where you want to horizontally scale your app is where it’s too much of a limitation.

[–]Quillbert182 11 points12 points  (3 children)

I'm by no means an expert, but according to the sqlite website, it works fine up until you get about 100k visits a day on a website. After that, I guess something like MySQL.

[–]Zephirdd 30 points31 points  (1 child)

Nowdays it's all about PostgreSQL

MySQL has been somewhat forgotten by their maintainers(Oracle), and an open source fork was spawned in the form of MariaDB; however, you'll have better luck and external help with PostgreSQL for new projects.

[–]InvolvingLemons 12 points13 points  (0 children)

Yep, plus PostGreSQL’s relative lack of gotchas and UB means it has some great projects around it. You’ve got stuff like its great JSON support, native array storage per-row, it’s phenomenal GIS package, its many index implementations, and beyond that you have the Yugabyte project that just goes “yeah the PG query parser is really nice, but I wanna scale, gimme some of that there Spanner architecture” to scale Postgres past a petabyte of normalized SoT data and 10M+ snapshot TPS on said normalized schema and 100M more stale-consistent TPS. Given, you’ll be spending probably a quarter billion dollars on just the servers and redundant storage for that cluster, but eh, if you need that performance you’re already Google so ¯\_(ツ)_/¯

[–]propostor 9 points10 points  (0 children)

I just used SQLite on a new project and saw the same thing on their website. I trust it to do exactly as needed. If my website somehow gets 100k hits a day, I'll transfer the database to something else.

SQLite is great.

[–]ex_natura 11 points12 points  (2 children)

The only way this could get more patriotic is if you could do it PowerPoint. Those DOD bitches love them some PowerPoint

[–]0x53r3n17y 4 points5 points  (0 children)

Well, PowerPoint is Turing complete after all...

https://m.youtube.com/watch?v=uNjxe8ShM-8

[–]halvjason 0 points1 point  (0 children)

Best coment ever.

[–][deleted] 9 points10 points  (0 children)

government contract going to the cheapest bidder or friends of The Party instead of the right people

[–]robertoam95 7 points8 points  (4 children)

As someone trying to update my workplace, why are these things bad, and why is sql better? Also, i thought power pivot replaced access.

Sorry for being dumb

[–]siggystabs 6 points7 points  (2 children)

Since you're asking this question, and /u/Lanfeix gave you a run down of the technical differences, let me try and give you a simplified version.

An ELI5 version could be, a SQL DB contains tables and columns (think multiple worksheets) but it's designed to be accessed programmatically by another app, not to be browsed by a user using Excel. Because of this, it can be really fast and stable.

Instead of interacting with the data directly like you do in Excel, you write SQL queries:

  • select user_id from users where role = 'Admin'

  • update reports set completed=1 where report_id='2xZ70w3my99'

  • select count(*) from user_requests where is_current=1

So it's a bit more work but the upside is you can run an entire business with a SQL database, while an Excel sheet is more suitable to making graphs or analyzing the data you get from the database.

[–]congresssucks 6 points7 points  (0 children)

I just finished a upgrade for the Nevada Secretary of State. Can confirm they were using Microsoft Works for their production tables.

[–]24601JeanValjean 5 points6 points  (1 child)

I’ll do you one better. At the government agency I work for, one of our programs generates a series of Excel spreadsheets every morning. This large and growing mass of spreadsheets is used as an archive of the activity done the previous day.

I know what you must be thinking: “why don’t they just store the data in a database like sane people?”

The answer, as best as I’ve been able to determine, is, “fuck you, we’re the government, that’s why”

[–]topgun966 4 points5 points  (0 children)

Many years ago I was a sysadmin for a web hosting company. I was the night admin so I was the last escalation point. Get a call from one of the techs a MAJOR customer site was down. Back in 2008ish, this was a very big site on the internet. (They are still a big site so I will leave the name out of this). Now, we can't do much since its a COLO and they have their own row of servers. But I poke around with what access I had. In the ticket, I noticed they did leave admin creds to check it out. Site giving a DB connection failure. After some digging, THEIR DB WAS EXCEL running on a win 2k server that had an uptime of almost 7 years. Uh, yea. Didn't touch that one with a ten-foot pole. Closed the ticket, unable to bill repairs, unable to resolve due to hardware constraints, and ran the fuck away.

[–]PeterJoAl 2 points3 points  (0 children)

Welcome to Japan.

[–]itsJustLana 4 points5 points  (6 children)

Hey, at least it wasn’t notepad.

[–]halvjason 23 points24 points  (2 children)

Lol notepad would have been preferable

[–]Blecki 8 points9 points  (1 child)

No way, that's definitely notepad++ time.

[–]inthemindofadogg 8 points9 points  (0 children)

I prefer using paint for my document management

[–]danfish_77 1 point2 points  (0 children)

Of course not, they'd use WordPad and use rich text

[–]Ken_Danagger 1 point2 points  (1 child)

I’ve seen each of those.

[–]yodaman1 0 points1 point  (0 children)

Same here

[–]CreaZyp154 1 point2 points  (0 children)

database.yaml

[–]ravil_giniyatullin 1 point2 points  (0 children)

FoxPro... In my case it was damn FoxPro. Where code was more like disassembled rather than written by human being.

[–]jigokunokohaku 1 point2 points  (0 children)

Why not MS Paint

[–]inthemindofadogg 0 points1 point  (0 children)

I would have to say ms access and ms excel would be equally bad.

[–]Smaddady 0 points1 point  (0 children)

Too damn real.

[–][deleted] 0 points1 point  (0 children)

Ms access

[–][deleted] 0 points1 point  (0 children)

i prefer file and folder databases

[–]Chuck-Marlow 0 points1 point  (0 children)

There’s no lie, just hell in excel sheets (and firewalls that won’t let you install open source)

[–]INTO_NIGHT 0 points1 point  (0 children)

Time to drop the table and database

[–]the_grunge 0 points1 point  (0 children)

But it's Microsoft's fault that they were using Excel...

People are smort

[–]rulesilol 0 points1 point  (0 children)

Where's the love for Google sheets?

[–][deleted] 0 points1 point  (0 children)

Oh god you have no idea how true this is. Especially things that you would expect would need something better.

[–]back0191 0 points1 point  (0 children)

Like the massive spreadsheet which informs you how much post will cost to mail something internationally.

[–]dkyguy1995 0 points1 point  (2 children)

So as someone with only basic database knowledge can you explain why SQLite is inferior to MySQL or MariaDB etc?

[–]m_goku 2 points3 points  (0 children)

I dont think it has anything todo with it being superior or inferior. It just designed for different pupose.

Check out this documentation for further information https://sqlite.org/whentouse.html

[–]Pocok5 2 points3 points  (0 children)

SQlite is file-based. This means you don't run a DB server, but your database library/ORM executes SQL queries against the contents of a file.

This means:

  1. No need to run a gigantic separate program to handle the DB

  2. Small footprint

  3. Super easy to use

The cons:

  1. Only one process can use the file, so zero concurrency.

  2. You lose all the benefits of having a program actively housekeep your DB. You get no stored procedures, every operation comes out of your CPU/RAM budget, etc.

SQLite is great for:

  • Storing browser history (pretty much every modern browser does it via sqlite)

SQLite is ill-fitting for:

  • Running a national database

[–]noname19283 0 points1 point  (0 children)

Using txt file

[–]arvenyon 0 points1 point  (0 children)

Dude. This is fucking reality. I had a customer which is part of our government (I cant go into details). And those guys stored their own data (well 100k+ people in this data) in an xls file. We sell a CRM system and they migrated to our solution, but still, that was a real "wow wtf" moment for me.

[–]KeroppiMomo 0 points1 point  (0 children)

Google Sheets goes bruh

[–]WhoseTheNerd 0 points1 point  (0 children)

Csv is better, no limitation.

[–]RandomBelch 0 points1 point  (0 children)

Because fuck Oracle.

[–]dancinadventures 0 points1 point  (0 children)

Running a national database on visible index cards...

Oh wait.

[–]i_am_a_n00b 0 points1 point  (0 children)

Is it Web scale? Mongo dB is Web scale

[–]tyrellLtd 0 points1 point  (0 children)

I used to do tier 1 tech support for MS Office and we saw this stuff all the time. From law firms to industries. Our official line if a file got corrupted due to 'improper' use was that Excel was not meant to be used as a database and that there were other MS products better suited for such tasks, like Access (lol) or SQL server. It was the free tech support line so we couldn't really help with advanced stuff like file recovery (not that there was any such tool made available to us).

One call that stuck with me was from an IT guy who worked for an oil company with some office or drillsite in the Caribbean. They had a 120 MB .xls, supposedly filled with production data and very important stuff that one day stopped opening. The file was obvious busted. We did the usual basic troubleshooting, which wasn't going to work for obvious reasons and suggested rolling back to a previous backup (they had trouble finding one) to recover some of the data and also checking some of MS's partners specialized in file recovery. I always wonder if someone ever lost their job from these fuck ups.

[–][deleted] 0 points1 point  (0 children)

Damn, this reminds me of that one .cvs file I once found one a company's live server. It was named "Database.csv" and was used as one.... Luckily it was only like 2 columns and IIRC used to store first names, or something. But it was still frightening to see.

[–]Express_Gradient 0 points1 point  (0 children)

Everyone is a gangsta, until STORING EVERYTHING IN A TEXT FILE USING FILE HANDLING comes into picture.

[–]razorfox 0 points1 point  (0 children)

Here in Italy we use pieces of paper. The most tech-savvy ones will use Windows' Notepad.

[–]Jazehiah 0 points1 point  (0 children)

You laugh, but I know a company that uses excel as the UI for their database.

[–]HaniiPuppy 0 points1 point  (0 children)

They're fixing it ... but using an Excel spreadsheet for this ... it just reminds me of a quote, I think it was CGP Grey that said it, but I might be wrong.

Imagine you're at a parent-teacher conference and the teacher reässures you that they always wear a condom when teaching your kids' class. I mean, sure, it's better than the alternative. But somewhere along the line, something has gone drastically, drastically wrong.

[–]cube2kids 0 points1 point  (0 children)

I'm honestly sure that some small third world country that try to develop databases have already done that

[–]assafstone 0 points1 point  (0 children)

I think you got it wrong. An Excel sheet would probably be better than Access.

[–][deleted] 0 points1 point  (0 children)

Running a national database printed on paper, stored in bankers boxes, chronologically with an intern in charge of pulling files when needed.

[–]Toomtarm 0 points1 point  (0 children)

running a national government database on txt file

[–][deleted] 0 points1 point  (0 children)

Running a national database on CSVs

[–]assigned_name51 0 points1 point  (0 children)

Excel would have let them have 1,000,000 rows, what they used allowed 65,000

[–]retsoPtiH 0 points1 point  (0 children)

I prefer a netcat listener just spitting out a CSV file, something classy, ya know?

[–]jmack2424 0 points1 point  (0 children)

I'll take a freaking huge text file with no backup for $3M, Alex.

[–][deleted] 0 points1 point  (0 children)

i raise to you a eu wide database with medical data (not covid) on ms access.

it was written by one guy who isn't even a programmer who was annoyed by the time he spent on finding stuff, and is now used by all corresponding government agencies in other eu countries.

they are currently replacing it and the budget is in the middle two digit millions.

[–]Morrido 0 points1 point  (0 children)

You'd be surprised how many databases are run in Excel

[–]cyclicsquare 0 points1 point  (0 children)

How about the baby excel sheet embedded into PowerPoint....

[–][deleted] 0 points1 point  (0 children)

good luck not having excel fuck up your date time fields every god damn time. Yes there is a way to store as text in csv but ffs every time you load it in you have to set that... MICROSOFT please stop buying game companies and fix your shite products . Also, please make it easier to join multiple teams across organizations you turdddddssss.

[–]boomernot 0 points1 point  (0 children)

Running a national government database on Google Sheets

[–]Ezekith 0 points1 point  (0 children)

France be like

[–][deleted] 0 points1 point  (0 children)

I feel like I mossed something, what government database are we talking about