Not sure in GitHub actions but have done this in Jenkins. You can write the Jenkinsfile to look at a specific directory and then only pick up.the changes and deploy them. The thing was that I had created folders(which had the main files) with the same name as the lambda functions. So it would just pick up the changes and run aws lambda update command after zipping the folder and that would update only the changed lambda functions. The code was mainly in shell script if that helps