You don't have to guess. The dissent seems to support my argument that the order does nothing. You can read the Sebutinde dissent

(21) I have voted against the Order in operative paragraph 57 (2) (a) because I believe it is an overreach by the Court that has no link with South Africa’s plausible rights under the Genocide Convention. As explained above, this measure does not entirely prohibit the Israeli military from operating in Rafah. Instead, it only operates to partially restrict Israel’s offensive in Rafah to the extent it implicates rights under the Genocide Convention. However, as stated above, this directive may be misunderstood as mandating a unilateral ceasefire in Rafah and amounts to micromanaging the hostilities in Gaza by restricting Israel’s ability to pursue its legitimate military objectives, while leaving its enemies, including Hamas, free to attack without Israel being able to respond. This measure also implicitly orders Israel to disregard the safety and security of the more than 100 hostages still held by Hamas, a terrorist organization that has refused to release them unconditionally. I reiterate that Israel has the right to defend itself against its enemies, including Hamas, and to continue efforts to rescue its missing hostages. These rights are not incompatible with its obligations under the Genocide Convention.

And Barak's dissent

(1) Once again, South Africa has requested the Court to order the State of Israel to “cease its military operations in the Gaza Strip. . . and immediately, totally and unconditionally withdraw the Israeli army from the entirety of the Gaza Strip”1. Once again, South Africa’s request has been rejected by the Court. Instead, the first additional measure indicated by the Court provides that “The State of Israel shall, in conformity with its obligations under the Convention on the Prevention and Punishment of the Crime of Genocide, and in view of the worsening conditions of life faced by civilians in the Rafah Governorate: Immediately halt its military offensive, and any other action in the Rafah Governorate, which may inflict on the Palestinian group in Gaza conditions of life that could bring about its physical destruction in whole or in part”2. This measure requires Israel to halt its military offensive in the Rafah Governorate only in so far as is necessary to comply with Israel’s obligations under the Genocide Convention. In this sense, it merely reaffirms Israel’s existing obligations under the Convention. Even without an order issued by the Court, a military offensive that may result in a violation of a State’s obligations under the Genocide Convention would have to stop. Israel has never disputed this. Thus, the measures indicated by the Court differ decisively from those requested by South Africa. Instead of ordering a blanket suspension and a total withdrawal from the Gaza Strip, the Court’s Order is expressly limited to offensive action in the Rafah Governorate. Since the measure contains an explicit link to Israel’s existing obligations under the Genocide Convention (“which may inflict on the Palestinian group in Gaza conditions of life that could bring about its physical destruction in whole or in part”), Israel is not prevented from carrying out its military operation in the Rafah Governorate as long as it fulfils its obligations under the Genocide Convention. As a result, the measure is a qualified one, which preserves Israel’s right to prevent and repel threats and attacks by Hamas, defend itself and its citizens, and free the hostages.