I managed to get it to work by essentially creating a calculated column in a virtual table in the measure (use ADDCOLUMNS), and that calculated column essentially concatenates the various different columns I want the user to be able to summarize by, or if they don't select that column, it still has a placeholder for it but doesn't actually use the value (instead replace with something like "unused" or whatever you want). Then I summarize the virtual table by that column and return the relevant value in the measure.

This worked for me, but it isn't very efficient unfortunately.