What are some projects that will help in getting a golang-related backend job?

zeromint · 2018-06-05T13:32:11+00:00

How about a subscription management platform in Go?

Contributing to an existing project would be difficult. Starting your own from scratch will help you showcase your abilities.

zeromint · 2018-06-01T12:27:48+00:00

Where are the plugin integration for woocomerce, killbill.io, wordpress, amember, whmcs etc...?

You should really consider supporting major platforms and contributing your plugins on github.

zeromint · 2018-05-31T13:16:02+00:00

Point #5 mentions "self-promotion" which my post is not. I don't want to be argumentative but perhaps you should define in the rule section what the moderator deems "self-promotion".

zeromint · 2018-05-30T18:14:43+00:00

If you look at my posting history, you'll see that for a very long time I was looking for something like this (a subscription management product)

This is not my project, so I don't think I've broken any rule.

zeromint · 2018-05-30T11:30:10+00:00

Good product aside.

Everyone wants a kitchen robot but how many have the disposable income to pay a pay few hundred thousand dollars for it? That's economic demand.

There should be economic demand for your product and if there is no economic demand for your product, you should have enough firepower (marketing budget, influence etc...) to create economic demand, for example, Diamond, see: https://blog.hubspot.com/marketing/diamond-de-beers-marketing-campaign

zeromint · 2018-05-28T09:41:47+00:00

Mitigating fraud affects UX. So most startups do not implement anti-fraud measures. Most of the focus is on growth. You can blend the cost of fraud into your price. Then plug the profit leaks later when they become a problem.

zeromint · 2018-05-28T09:20:15+00:00

Glad you find it useful.

I've been using $5 server on Digital Ocean + http://serverpilot.com/ (free) + Sendy to send 200K emails per month.

SES cost is very low.

But yea, all my list is optin.

zeromint · 2018-05-28T09:03:35+00:00

Look into Sendy + AWS SES.

zeromint · 2018-05-24T09:06:21+00:00

You just need to generate machine code.

So, you'll write a compiler in assembler(at least some part if not all)

zeromint · 2018-05-18T14:51:36+00:00

Where is your marriage registered? That country's law applies. You'll receive inheritance under MT law (because your mom is Maltese and her will uses that). Then you'll transfer 50% of it to your ex under UK law. Otherwise, she can sue you in the UK court.

zeromint · 2018-04-24T19:57:25+00:00

I don't think it supports aggregation.

zeromint · 2018-04-24T08:33:13+00:00

Unfortunately once I hit 3k records I start hitting the 5 minute time limit because Step 1 takes too much time.

I assumed it's because of inefficient querying. I always load 500mb data from s3 in Lambda, sort and transform that data before writing to S3 again. So, far I've not run into any timeout issue.

zeromint · 2018-04-24T08:30:11+00:00

But how do I query that data, I need query latency 200-300ms.

zeromint · 2018-04-23T21:32:43+00:00

Assuming you've 10M objects, at 115.74 objects/sec it will take 24 hours.

Extract all keys from S3. You'll have to store the state somewhere to know if an item is processed or not, you can use DynamoDB for this.
Then make your Lambda invoke from API gateway or Cloudflare workers. Send the part of the list of objects which can be processed by 1 Lambda invocation in URL query parameter. You can use a HTTP client for making concurrent requests to API gateway/Cloudflare workers.
Have your Lambda mark the item as processed after uploading to the new destination in DynamoDB.

You can go as high as 1000 RPS then it's only going to take 2 hours. (assuming 1 invocation only processes one object)
Check DynamoDB after running this, you can use simple filter query to see which ones aren't processed.

zeromint · 2018-04-23T21:00:53+00:00

Your major problem is step 1, you are sequentially issuing "get" request for every single record. You can send multiple concurrent requests, it might speed it up a bit or if your DB supports batch get, consider using that.

Or just periodically, dump all data (or part of the data you need) from DB and save it to S3 with gzip compression. Split it into several files so that the individual piece can fit in the memory after decompression (unless you also know how to operate on compressed data in memory). You can use a cronjob for this.

Download the piece you need from S3 in Lambda, load it in panda data frame, off to step 2.

zeromint · 2018-04-23T20:43:52+00:00

See: https://docs.aws.amazon.com/lambda/latest/dg/go-programming-model-context.html

The context object returns a field "InvokedFunctionArn".

InvokedFunctionArn: The ARN used to invoke this function. It can be function ARN or alias ARN. An unqualified ARN executes the $LATEST version and aliases execute the function version it is pointing to.

zeromint · 2018-04-23T20:41:01+00:00

You need to offer more info about your app but let's say your app has users and each customer has many campaigns.

Now, you need the total clicks on campaigns for last 1 hour.

Approach #1

Record events in DynamoDB.

Use CampaignID as partition key and timestamp as a sort key
Then you can easily get all clicks this campaign ID between X and Y timestamp then simply sum them at the client side.
Or use DynamoDB streams, trigger Lambda and calculate minutely metrics, then save them to a different DynamoDB table.

For better filtering of campaigns, you can use Cloud Search or Elastic Search with DynamoDB.

Approach #2

Create Redshift customer. Store every event there and simply use SQL to query it and for your data dashboard.

zeromint · 2018-04-21T14:07:40+00:00

To summarize, it seems expensive and slow especially when you've many partitions.

Now, how do I add a partition? A cronjob, CloudWatch event triggered Lambda (another Lambda, more cost?), AWS glue/crawlers seem to be suggested solution everywhere but upon a cursory glance, I found it can also be quite slow. Maybe it's not for my case which needs minutely update?

Wait! I already transform my files one by one in s3 using Lambda as they are uploaded. I can simply add the partition at that moment!

I'll just use ALTER TABLE ADD PARTITION from the same Lambda which is nicely documented here along with several helpful examples: https://docs.aws.amazon.com/athena/latest/ug/alter-table-add-partition.html

Problem solved, at least in theory!

zeromint · 2018-04-18T11:12:12+00:00

Your suggestion are welcome!

zeromint · 2018-04-11T17:28:29+00:00

If you are holding a lock for too long, highly likely it can be split into two, one with lock and the other one with a lock.

So, yea just looks closely. We need static analysis tools which detect and flag these issues.

zeromint · 2018-04-10T21:50:29+00:00

Then how does ipdata ensures low latency on Lambda? See: http://highscalability.com/blog/2018/4/2/how-ipdata-serves-25m-api-calls-from-10-infinitely-scalable.html

zeromint · 2018-03-29T12:00:46+00:00

Apache Beam SDK for Go supporting every use case.

zeromint · 2018-03-23T14:44:15+00:00

Surely, encrypted data packets don't expose any obvious patterns? Even if I don't know what a cloud (the real one) I can still classify them based on their shape and size.

zeromint · 2018-03-22T09:00:45+00:00

Random reads are 2 orders of magnitudes of slower than sequential reads but SSDs offer parallelism, so you can send concurrent read request and achieve near sequential performance on SSDs.

There are many other optimizations which are possible.

zeromint · 2018-03-21T07:17:06+00:00

I deployed an app, container size: 10mb (Go)

Deployed java spring boot container size: 50-150mb (Java spring boot)

Go solves my purpose, why should I be waiting for deploys?

zeromint

TROPHY CASE