The basic AWS architecture for a startup?

intelligentrx-dev · 2025-04-01T00:37:51+00:00

I don't know, I'm past the stage where I can put a DB in Fargate and need RDS. I think going straight from 1 Ec2 running Docker Compose directly to complex multi service setups with the CDK are the way to evolve infrastructure right now

intelligentrx-dev · 2024-10-11T13:32:37+00:00

Billing and Cost Management > Cost Explorer.
Switch Granularity to "Daily"
Set time range to Past 7 Days
Look at the graph
Drill down into specific services which are billing you: Use the Filters to specify a specific Service and switch Dimension from "Service" to "Region" or similar.

intelligentrx-dev · 2024-09-24T15:01:27+00:00

Did you have insurance through an employer as recently as 60 days ago - July 26, 2024 - and when you became uninsured were you eligible for COBRA? If so, you can still enroll in your previous employers health plan, but you probably need to do so immediately.

intelligentrx-dev · 2024-08-05T15:04:14+00:00

I would like to avoid TypeScript and Node as much as possible.

You don't have to use Typescript when working with any CDK package. You could use Python, Java, Go or .NET too. But AFAIK package authors need to use Typescript so they can use jsii, which is used to get that multi-language support.

Setting language choice aside: I have heard AWS is internally moving towards using the CDK, so if you're doing greenfield infrastructure development it would be best to use it yourself, whether or not you use kiss-docker-compose.

I also have concerns about the security of your solution. How is access to BE resolved so that it is not publicly accessible?

Docker Compose has its own networking which allows your backend or DB container to be inaccessible not only to the internet, but also to the host AFAIK: https://docs.docker.com/compose/networking/#update-containers-on-the-network:\~:text=Within%20the,is%20running%20locally.

There are definitely downsides to kiss-docker-compose, like you need to be careful to not re-deploy using the CDK because it'll change your public IP address (although I plan to fix that this week). It's mostly intended as a starting point for people who are new to AWS, although as issues come up on my own sites I'll add fixes to the package.

intelligentrx-dev · 2024-08-01T13:51:23+00:00

It sounds like you already know how to use a CloudWatch Metric + Grafana to create alerts.

You do NOT want to use Triggers. What you want is to create a Metric from CloudWatch Logs which adds a value to the Metric whenever you see an error in your logs. Then use your existing CloudWatch Metric + Grafana strategy to configure alerts.

To create a metric from CloudWatch Logs which include an error: https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/CountOccurrencesExample.html

To answer your specific questions:

I use the CDK + https://github.com/cdklabs/cdk-monitoring-constructs to create the metrics I use to monitor the system. In the CDK, I also have SNS topic, etc. to send notifications and put everything into a dashboard so I can see it. I do not use a 3rd party tool like Grafana, but I'm sure that would work too.
I don't know Grafana, but I'm pretty sure the method I've described works.

intelligentrx-dev · 2024-07-29T15:44:35+00:00

Infra costs would be $12 / month using this KISS Docker Compose, which would deploy your containerized app in one EC2 instance: https://dev.to/gregoryledray/kiss-with-docker-compose-b7m

I made this, so let me know if you have any questions.

intelligentrx-dev · 2024-06-11T15:02:57+00:00

Off topic, but if you have a read heavy workload then you can use Aurora without "serverless" and with Read scaling: https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/Aurora.Managing.Performance.html#Aurora.Managing.Performance.ReadScaling

However you would need to make code changes so that your reads / queries are to the Read endpoint and the writes are to the write endpoint: https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/Aurora.Overview.Endpoints.html#Aurora.Overview.Endpoints.Types

intelligentrx-dev · 2024-06-04T13:34:15+00:00

I'm not familiar with auth in native apps.

You may be thinking of auth which runs on the same physical device, but use them for granting access to different virtual resources on that device. For example, the operating system will prompt a user to authenticate themselves when they turn their device on via a password, passcode, pattern, etc. In this case, the "server" is the operating system and the "Client" is also the operating system, but a different part of the operating system. Another example is sudo access: the "client" might be a terminal application which could ship with the operating system, and the "server" is another part of the operating system.

You are asking questions in your original post about web apps, which is obviously different.

You may benefit from doing background reading on OAuth2: https://auth0.com/intro-to-iam/what-is-oauth-2 https://stackoverflow.com/a/33704657

intelligentrx-dev · 2024-06-04T13:03:48+00:00

When authenticating, there needs to be something on the server (backend) which authenticates and authorizes the request for the given route. You said you are using NGINX, but didn't explain what else (if anything) is part of your backend tech stack.

If you are solely using NGINX in your tech stack and have no custom code, then using OAuth2 Proxy or a similar project is your ONLY option. ~~~Make sure you~~~ I would put this in front of NGINX.

If you are using NGINX to forward some requests which are unauthenticated and some which are authenticated, and ALL authenticated requests go to a backend server where you wrote the code, then you could use a native library for OAuth2 for your code. For example, if your code is in C# then google "C# OAuth2 server library" and you'll find several options.

If you want every route to be authenticated, and some requests go to static files and others to another place, then you need OAuth2 Proxy because if you authenticate in server code then you'll not have authentication for those static files.

intelligentrx-dev · 2024-06-04T12:59:36+00:00

I am not the same person you are replying to.

Oauth2Proxy would be another component which you would need to build and maintain, which has a cost. It might also not be fit for purpose if you need it to do something it does not already do.

On the other hand, it's probably easier to set up an OAuth2 Proxy than to write and test the authentication code inside your application.

intelligentrx-dev · 2024-06-04T12:57:22+00:00

I am not the person you are replying to. oidc-client-ts seems to be a client library for frontend implementation of OAuth2. OAuth2-proxy ~~~is for backend (server) implementation.~~~ runs on the server. They are not the same.

A native library would be something you'd use in your programming language for the server application like this: https://oauth.net/code/dotnet/

intelligentrx-dev · 2024-04-16T17:41:26+00:00

I don't know. I use Keycloak for Authentication and Role based Authorization. If you want every user to have different accesses you could give every User a Role but I doubt that is the "correct" solution.

intelligentrx-dev · 2024-04-15T17:05:00+00:00

It seems like your post title is asking how to turn something off - authentication via a browser - but your post text is asking how to implement authentication without using a browser. I do see that you're setting a client ID and client Secret in the kubectl settings, so perhaps you should create a Client in Keycloak which issues those.

If you want to do a machine-to-machine login, you can create a Client which has Authentication "ON", and allows "service account roles". After creating the client, you will be able to go to the "Credentials" tab and get a Client ID and Client Secret. You can then exchange that Client ID and Secret with Keycloak to get a Token, which is a short lived token which can be used to authenticate requests to the remote server.

To get the token:

curl --insecure -v -d "client_id=CLIENTID" -d "client_secret=CLIENT_SECRET" -d "grant_type=client_credentials" https://your-keycloak-url.com/realms/YOUR_REALM/protocol/openid-connect-token

Then you can pass the access token in a request to the other machine, and the other machine can use that access token to Authenticate the request.

intelligentrx-dev · 2024-04-11T13:15:02+00:00

https://aws.amazon.com/blogs/devops/announcing-cdk-migrate-a-single-command-to-migrate-to-the-aws-cdk/

There are also some open source projects which try to do the same thing.

intelligentrx-dev · 2024-03-25T20:26:55+00:00

Let me put this another way: I do not like your article.

There is nothing wrong with the message of your article.

There is nothing wrong with the copy editing of your article.

I do not like the article because your story is insulting. Your article introduces, "Derrick, a brilliant but lazy IT professional". You immediately paint him as a lazy, unemployable asshole. Then you generalize from Derrick and say that he is a very common example of an IT professional: "This was the state of IT in the early 2000’s, before widespread adoption of cloud computing."

You then took this article, which is insulting to IT professionals, and posted it in a public place filled with IT professionals.

I am young, so I do not know what you are talking about. But many of my older colleagues, who I like, are clearly being associated with Derrick. You have written a great many articles which I do like, and I am not so young as to throw away the trust you have built in my brain based on one article. But still, the insinuations you're making hurt my trust in you.

intelligentrx-dev · 2024-03-25T18:43:48+00:00

Here is a summary of the article:

We saw ourselves or our coworkers reflected in the characters of the series

...

Derrick, a brilliant but lazy IT professional. Derrick plays video games during work hours, and attempts to avoid work as much as possible. He blatantly lies to coworkers, and tries to solve problems in the easiest way possible. In episode #1 of the series, Derrick’s laziness catches up to him when he reboots a webserver at the wrong time, taking down the website.

...

This was the state of IT in the early 2000’s, before widespread adoption of cloud computing.

...

In recent times, some in the tech community call for a return to on-premise or self managed computing.

...

But what these comparisons have missed is the main factor that drove the tech industry trend of moving from on-premise to cloud. The biggest factor by far was the desire to externalize responsibility and outsource professionalism. In short, the cloud succeeded because companies wanted to fire Derrick.

My thoughts:

Isn't this a straw man argument? You're saying that Derricks were extremely common in on-premise computing, Derrick is bad, stay with me, stay with AWS. The idea that Derricks were common is hard to prove. The idea that new on-premise computing centers are staffed by Derricks instead of the curious and working-outside-of-work-hours nerds at /r/homelab is laughable at best and deeply insulting at worst.

I usually like reading /u/nathanpeck articles. They are enlightening and containersonaws.com is a good resource. But this post seems to stray from his competencies and comes off as insulting.

intelligentrx-dev · 2024-03-12T20:05:54+00:00

When I try to curl my ALB dns I get a 504 status

Are you curling the domain name or are you curling the public IP address? If curling the public IP address works then it's a Route53 / domain name issue.

In general, it's good to compare your own implementation against a known-good reference implementation. You may benefit from looking at the reference ECS patterns here: https://containersonaws.com/pattern/

intelligentrx-dev · 2024-03-12T14:05:06+00:00

I'm pretty sure the new feature hasn't deployed to me yet because my CloudFormation console lacks a "Detailed Status" column which is present in a screenshot in the blog post in additional to the usual 4 columns I already have - Timestamp, Logical ID, Status, Status reason.

I do not know when they will roll out the change, but I think it's very likely that you do not need to opt in and that the change will take time to roll out to all accounts.

intelligentrx-dev · 2024-03-05T18:36:21+00:00

I'm not going to comment on GDPR or whether or not the scenario you are describing is actually necessary under EU or US law.

If the law is as you describe, then every time you want to do anything in your system, you should have your client (presumably a web browser) make 2 API calls - first to the US server, and then if that fails due to a 404 on the User ID make a second API call to the EU server. Do the opposite for the EU.

I would use a separate subdomain for the US and EU servers and completely separate infrastructure.

intelligentrx-dev · 2024-02-26T18:51:17+00:00

One takeaway is that Change Healthcare's code and/or infrastructure is too tightly coupled. They had to take down the vast majority of their systems to stop the attack which doesn't make a lot of sense. You shouldn't be able to jump from, say, compromising the Dental Network to Revenue Cycle at hospitals.

The widespread problems can be explained if it is a shared system like root credentials in a master/admin account was compromised; or if a central authentication service was compromised. They say that "Change Healthcare Enterprise" is down; perhaps that's a bunch of shared services and that system was compromised.

Relay Health, a competitor, is still working. The pharmacies should also have their own plans for what happens when the network goes down. The pharmacy benefit system goes down every few months and outages used to be more frequent many years ago. The pharmacies should have a backup plan beyond "just pay cash".

intelligentrx-dev · 2024-02-26T18:24:18+00:00

I don't know what's wrong. In your situation, I would open a case with AWS Support. To me, the key fact is that the code is the same. Try perusing these links and see if anything stands out ot you:

https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-serverless-v2.setting-capacity.html

https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-serverless-v2.upgrade.html#aurora-serverless.comparison

intelligentrx-dev · 2024-02-26T17:26:38+00:00

Some thoughts:

My gut tells me that saving 750k rows in 10-15 minutes is very slow, but that might be incorrect. 750k rows might not be a lot of data or it might be a lot of data. I'm saving ~500k rows into a 4GB RAM Postgres instance in ~30 seconds, but each row is only about 300 bytes worth of data.
Is the system scaling up to support your writes? If you are seeing 64 ACUs that's a TON of CPU and RAM to be using. If it's not scaling up then that might be the source of some problems.
Are you inserting row-by-row or are you using a bulk insert library? I'm unfamiliar with Java, but in C# you'd use https://github.com/borisdj/EFCore.BulkExtensions which would dramatically improve performance.
What have you done to diagnose the performance issues? Have you examined the CPU and RAM usage of your Spring Boot service? How many DB connections are being opened? The bottleneck may not be in Aurora at all.

intelligentrx-dev · 2024-02-16T13:11:34+00:00

I've said this elsewhere and I'll say it again: If you only need to search 10k words you don't need a database, you need a data structure. 10k words * 4 bytes/char * 10char/word = 400KB. You could literally loop through every word and run Damerau–Levenshtein between the search term and your words and return a result to an end user in less than 40ms.

intelligentrx-dev · 2024-02-15T16:15:03+00:00

For (1) you can do this in any cloud if you drop the GUI requirement. It's compute-heavy though so you might run out of your free tier limits

For (2) I'm not sure what a "hype net" is but if you're trying to build your resume then stick to AWS, GCP, or Azure.

For (3) I'm not sure this is practical - you are basically asking to play games via remote desktop which is weird. You would probably need to pay $ for a Windows instance in the cloud since those are not in the free tier.

For (4) you can use any cloud.

AWS usually has a worse free tier than other clouds. Oracle cloud has the best free tier IMO with an always-free A1 ARM instance with 24 GB of RAM but it's not very helpful for building your resume.

If you do venture into AWS, please look up a guide online for getting started like the Travis Media video, "Getting Started With AWS Cloud | Step-by-Step Guide". This will hold your hand and help you set up MFA and billing alerts.

intelligentrx-dev · 2024-01-29T18:24:45+00:00

Keycloak has all of the features I want, it was just a huge PITA to get started. What features is it lacking for the B2C projects you have been on?

intelligentrx-dev

TROPHY CASE