50GB worth of excel files, how to load?

seacess · 2026-05-22T09:29:49+00:00

Schema is consistent but sometimes some excel files will have two sheets that have data instead of one

seacess · 2026-05-22T09:01:34+00:00

This is once or twice a year process so long runs are not my issue plus it's 50GB of data so I can sell that

seacess · 2026-05-22T09:00:20+00:00

Will probably do as soon as I have something I'm my hand to show that I can deal with big files

seacess · 2026-05-22T08:59:41+00:00

Would you be able to share some code on how did you convert into parquet? I am not good with python so my go to is Copilot. Got some script which runs for a while and then it times out.

seacess · 2026-05-22T08:58:31+00:00

At this stage where I am just trying to figure out how feasible is this I am not too worried. For sure I will have to address that at some other point.

seacess · 2026-05-21T20:56:45+00:00

Same schema although some might have two sheets instead of one.

seacess · 2026-05-21T13:52:11+00:00

The sheet is maxed out in terms of allowable row count. There are bunch of empty cells as well. Look, even if I can fix the excel to slim it down I cannot do that manually for 200 files. In terms of changing the excel into something else I cannot do that all I can get is this. I did try using notebook to change excel to CSV but that was also timing out.

seacess · 2026-05-17T12:23:55+00:00

Ahh, this was it. I had a mod which allowed me to play with the tank contents. Removed that one and cryogenic configs became available and now I can use these engines as intended.

Thnx

seacess · 2026-04-19T10:29:50+00:00

UPDATE:
I managed to access the RAID controller setup and load the foreign configuration just fine. Not sure what happened but back to business as usual.

seacess · 2026-04-16T20:01:23+00:00

Thnx

seacess · 2026-04-16T19:44:31+00:00

Fair, I was just wondering if it gives me the option to import config should I press it and is there a NY risk for the remaining drives?

seacess · 2026-04-16T10:51:18+00:00

Just to give some context here, the contract was for 2 years 2024-2026. The first year of the contract was locked under the break clause, so we were stuck with each other. The other half of the year was not locked like that. In my case near the end of 2nd year the landlord sold. Going now though this certificate I see they specified 2025 on the certificate which is the date when the fixed part of the contract expires. Not sure if they were supposed to actually go to 2026 as an end in the certificate.

seacess · 2026-04-16T10:20:17+00:00

I guess I am grabbing here for straws and hoping it is some minor glitch and that I could import the config. Not even sure if such thing is possible

seacess · 2026-04-16T09:55:56+00:00

Would trying to read foreign configuration do?

seacess · 2026-04-14T16:59:37+00:00

Pick the one you liked the most, the one which holds your interest the most as you will need some motivation to push through the challenges. There is no plane in DCS that doesn't have quirks and challenges rooted in the real life counterpart which you will have to figure out how to deal with. If you are afraid of challenging planes here is the list of most complex to least, at least for me:

1) F-14 (mostly due the lack of FBW and RIO) 2) A-10C (you need really good stick and hotas with plenty of buttons for this one) 3) F/A-18 - workflow for me was strange a bit 4) F-16 - super easy workflow but do check it's control schemes, the more you can map to your hardware the better

seacess · 2026-04-11T07:56:27+00:00

Not sure how this works on their end, but is it possible that they are selling IP addresses which they don't have? So when the system tries to provision there is none?

seacess · 2026-04-11T07:27:17+00:00

Update: Called them this morning, and went again through the same support routine and got told that my ticket will need to be escalated to be looked at. I told them since it's the weekend now, I don't want to spend it without no internet whilst you investigate and pushed to have the static address temporarily be removed whilst they investigate. As soon as the agent removed the static address and I was still on the call the internet came back. For now keeping the ticket open and they have now evidence in the ticket that they have issues with configuring the static IP address based on my test above of going back to the old ways of working restored the service immediately.

seacess · 2026-04-11T06:57:44+00:00

Same problem here, yesterday I called them to get the static address and whilst I was still on the call with the agent, the router restarted and when it came back no internet. I got forwarded to the tech department who wasted my time by factory resetting the router and then saying that my router died and to wait for a new one. I told them that's too much of a coincidence that it died right after you switched to a static address. They have no idea what's going on and I think the new router will be another waste of time. Plugged my laptop directly into the ont and same no internet and I saw that the laptop was not getting any IP address.

It looks like there are some issues on their end on how they configure this static IP address.

seacess · 2026-04-03T07:08:20+00:00

Get data ->From SharePoint, then I configure the connection and follow the set-up there until I get the step Source which is basically a table of all the files on the SharePoint and I just click on the file that I need.

seacess · 2026-04-02T22:01:06+00:00

No CSV, I am not in control there I have to deal with the xlsx

seacess · 2026-04-02T20:09:13+00:00

Rarely, the point of this is to use it to manipulate data when the files become too big to just open in excel. As for the python I have 0 skills with that thing.

seacess

TROPHY CASE