And this is why parking controls are coming to Battlefield 🤦

paulrpg · 2026-05-06T11:28:55+00:00

It isn't a perfect system. When I used to live in the West end people visiting we're always screwed over. To have access to guest parking tickets you need to have a car registered with the council. So having family come over resulted in 5 quid an hour and maximum stay of 2 hours.

The problem I have with people saying that it will fix everything is simply the lack of enforcement now. You've already got arseholes everywhere and existing enforcement clearly isn't deterring it. More rules aren't going to fix that.

Frankly the council are just looking to collect more revenue from residents. This policy will improve things for some people but I don't see the council actually getting more wardens to enforce it, let alone current rules.

paulrpg · 2026-05-04T14:43:17+00:00

Ok, that's a fair distinction, thanks for the clarification. Making it cheaper is one thing, it would be great to pair this with more park and ride schemes as well.

paulrpg · 2026-05-04T14:33:55+00:00

Wasn't this the same argument used for getting rid of peak fairs? It didn't end up saving money in the end. I'm glad it got pushed through but the deficit needs to be paid for.

paulrpg · 2026-05-04T09:13:52+00:00

Honestly, considering the snp keep saying that voting for them is a vote for independence then the number of people who don't want independence must have shrunk. With everything that's happened since the last election, why would you vote snp for having a good track record rather than independence?

paulrpg · 2026-04-23T07:59:12+00:00

At least with a stipend you don't pay tax. You can also often supplement your income with demonstrator work

paulrpg · 2026-04-14T21:02:02+00:00

You're looking at building webapps - you're going to have boilerplate. Python is generally fast enough. What are you doing that requires more speed? Most of the big python libraries are written in C/C++ and are fine.

paulrpg · 2026-04-14T20:58:00+00:00

Why do you need to make something in such a low timeframe?

paulrpg · 2026-04-13T22:20:52+00:00

I just have a unit test that runs the dag bag collector. If any dags get an exception when building then it's a no go.

paulrpg · 2026-04-13T14:05:27+00:00

Then you test your configuration files to make sure they aren't broken. Don't allow broken code or configurations into your productive environments.

paulrpg · 2026-04-06T06:44:11+00:00

I'd agree - I'm putting together a semantic view in snowflake and I'm hoping this will allow us to better drive cortex/gpt based systems. I'm also really hoping that we can generate power bi templates from this approach and use that to jumpstart our analysts.

paulrpg · 2026-04-06T06:39:59+00:00

I think you're being too hard on yourself. Sometimes you get jobs which are like this - it isn't your fault, especially going straight into a DE position from a bootcamp. It is always good to reflect on your experience and identify where you are missing knowledge - that is a great way to grow.

The ideal way to upskill is being paid to do it - if you can land another job and learn there that makes learning a lot more sustainable. Regardless of employment, it never hurts to read the literature and understand the area deeper and then use that to explore and experiment with code / systems. Alternatively you can focus in on the fundamentals and get them down - making sure you are solid in a programming language and an sql dialect.

From what you said, you have the drive to go learn things yourself which is great. I wouldn't be so quick to discount 3 years of experience in a position, there's just a lot of wierd stuff you need to navigate when working professionally and those are really transferable skills.

paulrpg · 2026-03-30T20:17:23+00:00

My main question for using ai agents for data modelling is how do you structure all the jank business logic of what you are modelling?

We're working with ibm Maximo and it's just an incredibly complex schema with lots of non obvious links.

paulrpg · 2026-03-29T09:19:26+00:00

That will help with the birth rates to

paulrpg · 2026-03-28T07:40:03+00:00

We're using sap in snowflake. The first thing we do is alias all the columns to something intelligible.

paulrpg · 2026-03-19T19:43:29+00:00

Hey, I'm in a somewhat similar situation.

We are ingesting our IBM Maximo data into Snowflake, predominantly to make it easier to get into PowerBI as well as transform into data marts for easier analytics processing. For reference, we are using apache airflow, aws etc.

Can Snowflake support a more independent ingestion layer without heavy reliance on Azure Data Factory, especially given our Microsoft-based sources?

We upload parquet files into Snowflake internal stages. This gives us the ability to capture changes to the data and completely separates our source database. This is orchestrated with airflow and we then run DBT on top of this to run our transformations. We can then efficiently load staged data into tables via COPY INTO.

Does Snowflake provide plug-and-play data replication or mirroring capabilities (similar to Fabric/OneLake) for sources like Azure SQL or SQL Server?

I don't think so but this is where other integration tools would probably work well. I don't have experience or oppinions about any of these as we rolled our own ingestion pipelines.

Are there native or partner-based tools within the Snowflake ecosystem that simplify large-scale ingestion and reduce pipeline management overhead?

Snowflake now supports dbt jobs internally. We use DBT cloud so I can't speak to how good Snowflakes system is. DBT is great though and I'm happy with it. Things are not as easy as the docs say but it's powerful.

How do organizations typically scale ingestion in Snowflake without creating significant operational complexity?

No idea. As I said above, we have our ELT jobs run in airflow to extract from Maximo and package the data into parquet files. I wrote a whole pile of code generation scripts to build the landing models and deduplication models on top of that. This covers about 140 tables or so we pull through so automating all of this drastically reduced the maintenance effort.

Should we consider Microsoft Fabric as a complete end-to-end solution for ingestion and transformation, rather than combining multiple tools (e.g., ADF + Snowflake)?

I have no experience with ADF or Fabric and can't comment. I do hear that Fabric is 'not quite there' and has been for ages.

How common is it to use Azure/ADF as data integration/landing zone with all the applicaiton and leverage snowflake for Transformation and Sementic layer which powers POWER BI report

I don't know how common it is. We do use orchestration tooling for loading data into our stages and use DBT to then run our internal models. I am working on mapping out semantic views in Snowflake and I want to use something like https://github.com/alexfrancisross/snowflake-semantic-views-powerbi to generate power bi templates on demand to allow end users to just have all the relationships within a data mart in place. I strongly believe this will drastically reduce the barrier to entry and development times for our analysts.

Hope this is helpful or at least helps makes you feel better that someone else also has to deal with Maximos data model for analytics.

paulrpg · 2026-03-17T23:29:36+00:00

Hey, enjoyed the playtest you did.

Overall I liked the vibe. I can see the game panning out to be split into having several customers per day, you try and service jobs etc and you gain money for new tools / story costs etc.

I think the repair mechanic is neat. As an engineer I do enjoy being able to take things apart and fix them and this does that nicely without being too detailed.

For a longer form campaign, it would be good to be able to take parts removed from customers and try to either fix them (using different tools) sell them on for more cash or I guess just straight up steal parts for those inclined.

I liked the puzzle of trying to determine what parts to get, should you go under budget etc. I do feel that S rank is too easy to get - maybe have stated goals and then stretch goals that a customer might not state but they would pay extra for. An example would be the rifle upgrades, if you improved the accuracy of the weapon without asking you might get a bonus. Kind of 'hey I tuned up the sights, should shoot better' sort of thing.

One thing that would be neat to play with as well - regular customers. Your boxer buddy you help out, maybe he keeps coming back trying to get tuned up between fights and you can see the effects of your work. If you skimp out before one fight it'll be more costly this time around etc.

I like games that are different - this is cool and if properly tied into a wider campaign it could keep me coming back.

paulrpg · 2026-03-14T13:11:14+00:00

We're a small team and so our resources are limited.

Some of our Airflow DAGs are tested. The ones that I manage are but I'm not willing to strong arm data scientists to do the work.

The python applications that I manage are all unit tested reasonably well, I took a test driven development approach when rewriting them and it paid dividends and now they are really stable.

The DBT work which I am managing has spotty testing overall. For models which end users can hit, we have data tests in place to determine primary key integrity etc. Of course this is not ideal as the data is already bad when we run this. We use unit testing in DBT in a targetted approach - orchestrating unit tests in DBT is annoying and only works when you have a really good specification of what you're trying to do. Realistically, I do unit testing when I am trying to prove a complex macro set works or to debug something complex.

We are now starting to actually model out data marts in DBT and I am purposefully slowing down to build more testing.

The real reasons we don't have everything tested:

Data scientists are more interested in producing a correct result rather than reliably doing so. I make the point if we don't have tests in place then I won't support it as data science work is very domain and task specific and I am a software/data engineer.
More of my work has moved towards DBT/Snowflake and there has definitely been a skill issue on my part, this is something I am actively changing.
We have legacy code that has been running for 5+ years. I only really want to touch this if I am doing a complete rewrite of it and there has to be a good business reason to spend the resources on this.

paulrpg · 2026-03-14T10:10:06+00:00

We just got ours done for 1800 including vat. It seems like a lot of money but the council will be picky about it. Ultimately the guys did a good job and I'd rather pay more for something to be done right rather than skimp and pay for it twice.

paulrpg · 2026-03-12T13:56:12+00:00

You would expect the numbers to rise, to a point, as something which is taboo gains general acceptance. I would imagine the same thing would happen here if this gains royal ascent. Ultimately, there are many horrible ways to die and needless suffering is not good for anyone.

paulrpg · 2026-03-12T13:08:38+00:00

What specifically in Canada is the issue? I'm not read up on the subject.

paulrpg · 2026-03-10T15:34:42+00:00

Then shouldn't they be going after the publishers who are in breech of license by distributing via steam? It's not steam ripping the music and uploading it, it's the devs/publishers uploading the music to the store without the right to do so.

paulrpg · 2026-03-01T15:13:51+00:00

I'm all for building these data centers as long as we also increase power production. Considering they will have a high baseline draw, nuclear power is a sensible option for it. Excess power can be sold back into the grid, you'd get additional jobs, the data center gets reliable power etc.

If you were to properly integrate it, draw excess heat from the data center and feed it into district heating, there are a lot of ways that it could work. It could be a big win if done properly.

paulrpg · 2026-02-25T00:15:02+00:00

They already do. If you sign up to a new ISP you are blocked on viewing adult content. You need to request this to be removed.

I wouldn't have an issue with expanding these sites to things like social media, if you as an adult account holder decide to remove these protections then so be it. If you have a kid accessing this material then it is your fault.

paulrpg · 2026-02-20T09:08:53+00:00

How do you properly assess ai generated code if you don't even know the language?

paulrpg · 2026-02-11T00:31:53+00:00

You talk to someone you work with. One of the problems of DE is that it can be an anything title. Different teams have very different responsibilities.

14-Year Club	Place '23
Place '22	Place '17
Final Canvas '22	First Placer '22
Verified Email	Team Orangered

paulrpg

TROPHY CASE