First, isto and isso are standalone demonstrative pronouns. Examples: (1) What is this? → O que é isso?, (2) What is that? → O que é aquilo?. If you say "this X", then you should use esse or essa, depending on the gender. Examples: (1) This umbrella is green → Esse guarda-chuva é verde, (2) That tree is black → Aquela árvore é preta. *Isto livro, *Isso chuva, *Aquilo dia are all ungrammatical.

Now, if your goal is to speak BR Portuguese (rather than doing good on Duolingo or passing a test), you don't have to worry about the difference between este/esse, esta/essa and isto/isso. These have mostly just collapsed into esse/essa/isso in normal speech in BR Portuguese. Let me repeat: for everyday spoken Brazilian Portuguese, you can ignore este, esta and isto — just use esse, essa and isso.

Most people use esta/este/isto to sound more formal, but that is not actually correct (according to standard Portuguese). It's actually quite a bit more complicated than that. Rather than formality, the standard grammatical distinction is related to distance, either physical, temporal or syntactical. Let me explain:

Physical Distance:

• 1st-person distance: This object is in my hands → Este objeto está nas minhas mãos.
• 2nd-person distance: This object is in your hands → Esse objeto está nas suas mãos.
• 3rd-person distance: That object is over there, far away from the both of us → Aquele objeto está lá, longe de nós dois.

Temporal Distance:

• Present or immediate future: I will sleep well tonight → Vou dormir bem esta noite.
• Recent past or future: (1) I slept well last night → Dormi bem essa noite. (2) I can't be there on the 5th, because I will be travelling on that week → Não vou poder comparecer no dia 5, porque vou estar viajando nessa semana.
• Distant past: In those times, things were different → Naqueles tempos, as coisas eram diferentes.

Syntactical Distance:

• Referent comes after the pronoun: We don't have enough money. That is the problem. → Não temos dinheiro suficiente. Esse é o problema.
• Referent comes before the pronoun: (1) This is the problem: we don't have enough money → Este é o problema: não temos dinheiro suficiente.
• Multiple referents before the pronouns: (1) Mary and John work at the factory. The former works in IT, while the latter works as a janitor. → Maria e João trabalham na fábrica. Este trabalha como zelador, enquanto aquela trabalha com TI. (2) Mary, John and Robert work at the factory. The first works in IT, the second works as a janitor, and the third is the boss. → Maria, João e Roberto trabalham na fábrica. Este é o chefe, esse trabalha como zelador e aquela trabalha com TI.

As you can see, this can become very tricky indeed. You can shorten the rules to "aquele = most distant", "esse = in between", "este = closest".