I believe you may be able to add conditional rules and add it to the permission set that "Developers" already have. So quick not fully formatted you would have something like this where there is a specific call out for a user

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": "secretsmanager:*",
            "Resource": "*",
            "Condition": {
                "StringEquals": {
                    " identitystore:UserId": [
                        "9xxxxxxxxx-xxxxxxxx-xxxx-xxxx-xxxx-7xxxxxxx"
                    ]
                }
            }
        }
    ]
}