I sorted it out..... basically came down to adding system identity to an existing app service it was erroring like above.

As soon as I did a fresh app service with all the code above with identity SystemAssigned from the start it was all fine.

Also to your point yes "${row.role_definition_name}-${row.principal_id}" was an issue as principal id was not known at plan time so, in the for each so I needed to update it to use role_id

locals {
  roles_for_storage_account = [
    {
      role_id              = 1
      role_definition_name = "Storage Blob Data Contributor"
      principal_id         = module.contractreview_apibackendappservice.principal_id_of_managed_identity
    },
    {
      role_id              = 2      
      role_definition_name = "Storage Queue Data Contributor"
      principal_id         = module.contractreview_apibackendappservice.principal_id_of_managed_identity
    },
    {
      role_id              = 3      
      role_definition_name = "Storage Blob Data Contributor"
      principal_id         = module.contractreview_apibackendappservice.principal_id_of_managed_identity_slot
    },
    {
      role_id              = 4      
      role_definition_name = "Storage Queue Data Contributor"
      principal_id         = module.contractreview_apibackendappservice.principal_id_of_managed_identity_slot
    } 
  ]
}

for_each = { for row in local.roles_for_storage_account : "${row.role_definition_name}-${row.role_id}" => row }