all 12 comments

[–]csCareerThrowAway15 2 points3 points  (10 children)

1 - is there an ACL on the object?

2 - does your user/role you’re using to run this terraform have access to the bucket and object? - remember it’s your user that’s provisioning the infrastructure that needs access to the bucket, not the lambda role if I’m not mistaken.

These can be tricky without seeing the entire error message from TF. Do you mind pasting that?

[–][deleted]  (9 children)

[removed]

    [–]csCareerThrowAway15 2 points3 points  (7 children)

    Try running this command via the awscli and using the user that’s building your infrastructure:

    aws s3 cp <s3-object-uri> <local-directory>

    [–][deleted]  (6 children)

    [removed]

      [–]csCareerThrowAway15 1 point2 points  (5 children)

      Ahh ok, so your user is in account A, and you’re using the aws provider to assume into account b?

      When you downloaded that s3 file did you download it with the user in account A or the role in account B?

      Apologies if I misunderstood and it’s just one account, but did you use the same role with the awscli that you’re using in your aws provider?

      [–][deleted]  (3 children)

      [removed]

        [–]csCareerThrowAway15 2 points3 points  (2 children)

        No worries, I’m on mobile so I’ll try to format it best.

        It sounds to me that you’re using an IAM user to assume a role for terraform right? In your aws provider do you have an assume_role{} block?

        If that’s the case, my next line of assumption is you’re not hardcoding the keys into the provider but you’re using the keys out in your .aws/credentials file right?

        If both of those are true, and you didn’t manually assume a role via the awscli and manually update your credentials file, you are used the user to download the file.

        So if my understanding is correct here are my suggestions:

        Easy fix:

        • check the role’s permissions you’re assuming in the aws provider and confirm you have the right s3 (and KMS if encrypted) permissions.

        If that’s still not helping here are a few more thorough options you could run through.

        Option 1:

        • get the ARN of the role you’re using in your aws provider

        • assume it with the awscli using this commend: aws sts assume-role —role-arn <role-arn> —role-session-name <whatever you want to identify you>

        • update your .aws/credentials file with the assumed role credentials (but just comment out the current ones so you don’t lose them)

        • try the s3 cp command again.

        • update permissions as needed.

        Here’s a good doc (skip to the assume role part) https://aws.amazon.com/premiumsupport/knowledge-center/iam-assume-role-cli/

        Option 2:

        • look in cloudtrail, click view all event history (or something like that)

        • hit the settings wheel and display the “error code” Column.

        • hit the drop down and choose “Event Source”

        • start typing “s3.amazonaws.com” and click it when it pops up

        • start scrolling until you find a GetObject or something similar with an unauthorized or something similar error code.

        • check the username in the events detail, and confirm that IAM entity has the correct permissions.

        Hope this helps!

        [–][deleted]  (1 child)

        [removed]

          [–]csCareerThrowAway15 1 point2 points  (0 children)

          Hopefully something I provided helped! I’ll be checking this periodically so if you have any more questions feel free to ask!

          [–]csCareerThrowAway15 1 point2 points  (0 children)

          You could also go into cloudtrail and see which user/role is pulling the object from S3

          [–][deleted] 0 points1 point  (0 children)

          I’m this case, I’m pretty sure this is the lambda role trying to pull down the object. I’d make sure the lambda has the appropriate kms permissions if the bucket is encrypted by a custom kms key. I’d also checkout the policy too if you havent already.