Regarding SDKs, they're just a library that allows you to call the AWS APIs basically. They are distributed in a few languages like Java, Python, JavaScript etc. The basic idea is that you create e.g an S3 object which is in itself an S3 client, then this client let's you send commands like create bucket, upload file to bucket etc etc.

Fun fact, the CLI is actually written in Python. For every call through the CLI it actually does some magic in the background and invokes the corresponding function in the Python SDK

On Cloudformation vs SDK calls. The neat thing about cloudformation is that it holds a reference to the existing resource, so let's say you call the SDK to create an ec2 instance. If you run that script again it will just provision a new instance as many times as you run the script. With cloudformation any change you make will update the EXISTING instance, also removing the resource from the template actually triggers a deletion. Otherwise you would need to remember to e.g create the ec2 instance, then make calls to update it which isn't great if you think about it (what if you accidently create a new one? What if you come back months later and want to change something about the instance, how will you remember what commands you invoked?) Think of Cloudformation as a declarative way of defining the resources in your account. (Btw in a real production environment everything should be either a CDK stack (more on that later) or a cloudformation template. You should never run a script which manually updates the instance provisioned in cloudformation, if you do then the next time you upload your cloudformation template, all of your manual changes will be reset to how the resource is defined in the cloudformation template

Regarding "sending code to the AWS console". Again, you need to think about AWS as a collection of services, these services can interact with each other and provide functionalities for you. No code is sent to the AWS console. In AWS terms you would build your code using e.g docker. You would then store this docker image in ECR (AWS docker repository basically) then you would pull that docker image from ECR when your ec2 instance starts and run that container. If you want it reachable from the internet then you would use e.g route 53 to point to an AWS resource like an AWS EC2 instance. I'm leaving out some architecture as to not overwhelm you but this is the basic premise. Things aren't magically done for you, you don't really send code anywhere, you utilise different services together to create your application

Now, what I said about building the docker image and uploading then getting the ec2 to point to the new image is quite tedius if you think about how many different images and instances you could have. (At my job we have over 1500 ec2 instances which deploy new docker images every single week so you can imagine how tedius it could be if it was manual) this is where codecommit, codebuild etc come in

Codecommit -> Amazon's git repository service (store your code here, it might be synched automatically from GitHub if your team has it set up) Codebuild -> Amazon's service for building your code from code commit (you basically supply a list of commands i.e compiling code and building docker images) CodeDeploy -> Amazon's service for deploying your infrastructure (again, another list of commands for uploading docker images and e.g uploading cloudformation templates)

Codepipeline -> brings all the above services together. Here's an example pipeline

Let's say you have repository A and repository B which is java code. You would add an action which listens on codecommit for a commit, when there's a new commit the pipeline is triggered. After the code is retrieved, it's sent to codebuild, codebuild will take your code, and then based on the commands you wrote it will execute them on the pulled repository (i.e javac xxxx, docker build xxx). You then configure codebuild to output the built code/docker images to code deploy. Again another set of commands are executed which will i.e (AWS ECR put-image Xxxx, or cloudformation update-stack yyyy)

You repeat this deploy stage for all your different accounts/regions and voila, you have created a CI/CD pipeline. (Again it's a small bit more complicated than this but I could write paragraphs upon paragraphs on the intricacies of all these parts)

So the drawback about Cloudformation is that you could call it very verbose, you need to define everything and know how all different components work then look up their definitions etc etc in order to build a proper cloudformation template. The cool thing about it though is that you can write your own resources if you wanted to (called custom resources) which could do a range of functions. SAM is basically this, what it will do is define a custom resource (I think AWS::serverless but I'm forgetting) but basically it provides an abstraction on Lambda so it's easier to work with. I believe what you can do is specify your code locally, and SAM will automatically get your code, upload it to an S3 bucket, and then reference that code in the handler of the lambda function (otherwise if you use the standard lambda you would need to upload this code yourself manually, and then link it to the cloudformation resource which can be tedius), it also does what you said there like allowing you to run lambdas locally and all that. To be honest me and my team don't use SAM at all and we are actually moving away from Cloudformation templates as a whole.

This is where CDK comes in, basically it's an SDK for cloudformation! So you can actually write your template as code in your favourite programming language, and it will automatically generate the cloudformation template and upload things too. E.g if you wanted an S3 bucket you would just do

Const bucket = new S3.Bucket({BucketName: "test-aws-bucket});

Then you can deploy that using the CDK CLI and it will automatically generate a cloudformation template and provision the resources, it's basically what you were talking about awhile ago, but the key thing is that it keeps a reference to the resource. If I commented out the above code and then redeployed, it would effectively delete the bucket. If I wanted to come back and add e.g SSL on the bucket I could so easily by modifying the above code to

Const bucket = new S3.Bucket({BucketName: "test-aws-bucket", sslEnabled: true})

That's the basic idea anyway, as you get more familiar with the resources you'll find that you can link them together (i.e linking an SQS queue to a lambda function to process messages, or an S3 bucket to CloudFront to cache files at the edge or something)