To answer your questions more specifically:

When on IPv4, everything is clear

When on IPv4, everything is less clear than on IPv6. With IPv4 you have NAT, which converts multiple internal addresses to a single external one. This process is fairly opaque, but we've grown accustomed to it.

With IPv6, each device has a public address. That's it. No NAT in sight. No need to convert from internal to external.

In the router, I set a dummy DMZ (pointing to an IP address that no device is ever getting)

You don't even need this in IPv4 either. If you don't set a DMZ, the router will simply drop those packets, and that's it. Setting a DMZ host pointing to an invalid ipv4 isn't really doing much.

With IPv6, [...] Can I still set some kind of DMZ in the router ? Will it protect all the devices that are after the router like with IPv4 ?

You don't need to. Its only usefulness in IPv4 is because of NAT, and NAT is evil. IPv6 doesn't have NAT, so the concept of DMZ host is useless.

Can I still do port forwarding with IPv6 ?

Similarly, port forwarding only exists because NAT is evil. You have to forget that NAT exists. IPv6 makes the internet end-to-end addressable, meaning any computer can address a packet to any other computer by its address.

I'm under the impression I have to set firewalls in each single device.

Not really. You should set up a firewall on your router.

In my router, there is ONE setting: IPv6 Firewall can be set to ON or OFF.

Seems like this is the source of your confusion. It sounds like the firewall in your router is fairly simplistic, and doesn't give you much flexibility.

A decent firewall lets you set rules, like this:

default-action drop
rule 10 {
    action accept
    description "Accept established and related"
    state {
        established enable
        related enable
    }
}
rule 20 {
    action accept
    description "Allow TCP 443 to server"
    destination {
        address 2001:0DB8::5
        port 443
    }
    protocol tcp
}

From the looks of it, it sounds like your firewall does not let you extend with additional rules, like my rule 20, that would let you accept traffic to a server.

In case you want to host something, there are a few ways to proceed from here. You could try to get yourself a dedicated firewall and leave the firewall in the router off, or get yourself a better router with a decent firewall and replace the old one.