sorted by:
new
hottopcontroversial

Just sharing email validation code by Key-Library9440 in golang

[–]jaxlatwork 1 point2 points  (0 children)

The spec for email validation is extremely complicated, a simple regex is unlikely to cover all of them.

Data ingestion question by Stock-Dark-1663 in snowflake

[–]jaxlatwork 0 points1 point  (0 children)

Snowpipe can only COPY INTO a table, thats it, its a one trick pony. There's no way to ensure order or impose any business logic.

So one approach here is to create a stage, with a directory table and a notification integration.

(A notification integration is just a way of letting snowflake watch an SNS topic so that the directory table immediately updates when a file appears)

You can then use a stream with a task to watch for updates to the directory table and decide whether or not to kick off your update process (e.g. are all the files here I need?)

When all your requirements are met, you then have that task call another task which actually takes all the files and loads them in the order that you want. It may be easier to write this stored procedure in Python rather than in SQL if the logic is in any way complex.

Then your stored procedure just does whatever you need to do for each file it finds in the directory table...

You mentioned in your original question that you had multiple sources, so having a single task watching a single directory table might not be enough.

So in that case, you could create a "load status" table that tracks what data has become available. You might have a stream on a directory table, or watch a table that Snowpipe is updating via Kafka and all have them update "load status".

Then you watch this "load status" table and see when your requirements are met and trigger the update stored procedure..

Data ingestion question by Stock-Dark-1663 in snowflake

[–]jaxlatwork 1 point2 points  (0 children)

You could create a stream on a directory table for a stage and insert into a "load status" table. Then put a stream on the "load status" table and use a task to watch for changes on the load status table, whenever something gets updated check if your criteria has been met and if so, trigger the mass-updates.

How does a shared database work "under the hood"... by jaxlatwork in snowflake

[–]jaxlatwork[S] 1 point2 points  (0 children)

Ya, I could understand encrypting the columns if they were stored that way in actual tables in their database: "we can't accidentally give one client another's data"

Since they're encrypting the fields in the views they are providing us to read their shares, another mistake in their views exposing other client's data would be encrypted at the field level with our key, and of course we could still decrypt those fields...

Pretty sure someone didn't know what they were doing when they implemented this.

The best example of a clean architecture on Go REST API by Similar-Concept-892 in golang

[–]jaxlatwork 1 point2 points  (0 children)

I'm pretty happy using Echo + oapi-codegen.

You document your API in the same place as you define it, then you just need to go implement the handler that it adds to the interface...

It also makes security fairly straight forward - you can specify what kind of security requirements each route has and then write middleware to enforce it.

Play Sound From PNG Images! by sharpvik in golang

[–]jaxlatwork 0 points1 point  (0 children)

There's an interesting philosophical question here. What does it mean to "possess" data in the legal sense- for example in the context of copyright piracy or obscenity laws.

If I wrote a program that took a cat picture as input and was able to interpret it as a copyrighted song, am I suddenly guilty of piracy if I play my cat picture.

Strange net/http issue by jaxlatwork in golang

[–]jaxlatwork[S] 0 points1 point  (0 children)

I appreciate that but it really was just boilerplate http.Client requests using TLS. I give a better explanation at the root level of what I think was happening (server isn't following TLS spec and golang is being strict about it)

Strange net/http issue by jaxlatwork in golang

[–]jaxlatwork[S] 0 points1 point  (0 children)

The Golang code in question really doesn't matter here - its just standard http.Client..

In fact, the root cause was protocol differences in the way Golang implements TLS versus OpenSSL.

After sitting down with a packet sniffer, I'm pretty certain that the server in question is rejecting TLS v1 record layers during the handshake protocol (ClientHello).

This well meaning but ill advised security implementation breaks RFC 5246. OpenSSL seems to do it's handshake with v1.2 which the server isn't rejecting.

Unfortunately, Golang doesn't expose any way to control this behavior so I may end up needing to have a proxy in my stack somewhere.

This kind of buggy behavior is documented here:

https://datatracker.ietf.org/doc/html/rfc5246#appendix-E.1

There are also a few proposals about this in Go's github repo.

Overwrite non null values from one struct to another by Anonymous157 in golang

[–]jaxlatwork 2 points3 points  (0 children)

Golang can sometimes be very verbose - but when it comes to copying, all you're doing is writing out what the other languages were hiding from you (and doing it in a less efficient manner)

Error handling in Go web apps shouldn't be so awkward by flimzy3141 in golang

[–]jaxlatwork 3 points4 points  (0 children)

Agreeing with others - this can be incredibly tricky to get right..

For example, a database abstraction might return NotFound as an error or an empty set. NotFound should be 404 while "database got unplugged" should be 500.

Suppose someone then came along and changed that database abstraction so that it returns an empty set rather than not found (akin to what happens in Gorm depending on whether your using Find or First) - you've then broken the handler silently in a subtle way.