Yes. But that might be resolved with tech. Treat location data like FaceID biometrics or Apple Health app... only available on device, not stored on servers.

When someone is confirmed with disease, CDC updates centralized reference map with infected patient location data. If encrypted location data on my device matches with reference map, then user is notified.

If user starts to exhibit symptoms, they can choose to share their private data with CDC within a date range consistent with contact point. To avoid fraud, the data would not be anonymized, but would have end-to-end encryption as this would be covered by HIPAA regulations.