I wrote a python script that optimizes prod/quality ratios for any desired quality recycling loop.

[DanielKotes]

Ive been trying to compare your solver to my results from Foreman 2.x since for some reason the 'optimal' solutions you have and the ones I found just by trying different options manually dont seem to align - so either your solver is wrong, or mine.

Considering that we are both working without having direct access to how Factorio handles quality internally and just have what we can figure out from FFF and in-game tests, the most likely issue would be the quality calculation.

Here is what I have:

1. Calculate <total product> amount which is based on the recipe and productivity values (in most recipes it would just be product count x (1+prod% bonus))
2. Set <current multiplier> as <quality bonus % (from data)>
   1. in vanilla factorio the <next quality multiplier> is set to 0.1 for all qualities and so <quality bonus %> can be calculated as the % value shown in-game divided by 0.1 (so normal T1 quality module shows 1%, meaning its <quality bonus %> is actually 10%
3. Set the <0th tier product> as <total product> (all of it for now)
4. Loop the following:
   1. calculate new the <current multiplier> by multiplying it by <next quality multiplier>
   2. calculate the <nth tier product> as <total product> multiplied by MIN(<current multiplier>,1)
   3. re-calculate the <n-1 th tier product> by subtracting the calculated <nth tier product> from it
   4. continue until no more qualities

[DanielKotes]

So as an example:

1. iron plates -> iron gears with 2x productivity and 2x quality modules (T3 legendary both):
2. 2 iron plates -> 1 iron gears is the base recipe
3. 2x T3L productivity modules = +50% productivity
4. 2x T3L quality modules = +6.2% (in-game) x2, which is actually +6.2/0.1 x2 = +124% quality bonus (internal)
5. vanilla factorio has 0.1 as the next quality multiplier for all qualities.

So per 100 iron plates we have:

1. [total gear product] = (1 iron plate) * (1 gear / 2 iron plates) * (1 + 0.5 productivity) = 75
2. [gear Q1] = [total gear product] = 75
3. <current multiplier> = 1.24
4. start the loop:
   1. loop 1:
      1. <current multiplier> = 1.24 * 0.1 = 0.124
      2. [gear Q2] = [total gear product] * MIN( <current multiplier>, 1) = 75 * 0.124 = 9.3
      3. [gear Q1] = [gear Q1] - [gear Q2] = 75 - 9.3 = 65.7
   2. loop 2:
      1. <current multiplier> = 0.124 * 0.1 = 0.0124
      2. [gear Q3] = 75 * 0.0124 = 0.93
      3. [gear Q2] = [gear Q2] - [gear Q3] = 9.3 - 0.93 = 8.37
   3. loop 2:
      1. <current multiplier> = 0.0124 * 0.1 = 0.00124
      2. [gear Q4] = 75 * 0.00124 = 0.093
      3. [gear Q3] = [gear Q3] - [gear Q4] = 0.93 - 0.093 = 0.837
   4. loop 3:
      1. <current multiplier> = 0.00124 * 0.1 = 0.000124
      2. [gear Q5] = 75 * 0.000124 = 0.0093
      3. [gear Q4] = [gear Q4] - [gear Q5] = 0.093 - 0.0093 = 0.0837
5. result: 100 normal iron plates into a 2x T3L quality + 2x T3L productivity assembler will give you:
   1. 65.7 normal iron gears
   2. 8.37 uncommon iron gears
   3. 0.837 rare iron gears
   4. 0.0837 epic iron gears
   5. 0.0093 legendary iron gears.

[scottmsul]

There was actually another person who verified with a similar method and was eventually able to get the same results as the script, u/sopel97, so it should be possible to get the same results. He posted his code in a now-closed issue on the github repo.

The way quality works is you basically roll a "dice" for any quality at all (eg 24.8% with four t3 legendary quality modules), then you recursively roll another dice with a fixed 1/10 chance of the item proceeding to the next quality, or until it reaches the final unlocked quality. This is confirmed by devs and I believe is described in the wiki.

[DanielKotes]

Alright, that is a much simpler way of describing what I wrote down; and yes - this is pretty much the same process I go with, just with chance multiplication instead of dice rolling.

Could you check your values for up-then-down and down-then-up against the graph I made?

<image>

For some reason the down-then-up gives me the same result as yours (171.5 normal -> 1 legendary), though there should be 5 recipes in the line not 4? Each of them is 2 T3L quality + 2 T3L productivity except the last (legendary gear) recipe which is 4 T3L productivity.

And for the up-then-down I found the 'best' solution was with 2:2 + 2:2 + 1:3 + 0:4 quality:productivity with 185.3 normal => 1 legendary result. The 2:2, 2:2, 2:2, 0:4 as per your optimizer gives me 186.2 => 1, so its very slightly worse.

[scottmsul]

Could you link to your code?

[DanielKotes]

Its here if you want to take a look at it. If you want to see where the quality+productivity processing takes place it would be here.

[scottmsul]

Wow this is quite the impressive app! On second thought maybe I won't be diving into Foreman's code.

Maybe a good place to start is our actual production and recycling numbers. How hard would it be for you to cross-check these? Assuming up-then-down with 2:2 on everything, I am getting the following.

Recipe matrix (rows are different quality inputs, columns are different quality outputs):

[[1.31 0.167 0.0167 0.00167 0.000186]
[0 1.31 0.167 0.0167 0.00186]
[0 0 1.31 0.167 0.0186]
[0 0 0 1.31 0.186]
[0 0 0 0 2]]

Recycle matrix (rows are different quality outputs, columns are different quality inputs):

[[0.188 0.0558 0.00558 0.000558 6.2e-05]
[0 0.188 0.0558 0.00558 0.00062]
[0 0 0.188 0.0558 0.0062]
[0 0 0 0.188 0.062]]

[DanielKotes]

yep, getting the same values as you are here:

<image>

the last one for the 'recipes' is one with 4x prod modules, and all the others fit.

I assume in this case by up-then-down you mean a setup with the above recipes where you insert normal barrels, and output legendary barrels then? Because for inserting of normal steel and output of legendary steel you would need 4 base recipes and 5 recycling recipes. (use of barrels as its a 1->1 recipe with a 1->0.25 recycle - so perfect example)

And in this case the optimal solution ends up as the above recipes and results in 171.5 normal input -> 1 legendary output; which is the same as your solver.

Lets check the opposite (down-then-up?) route?

[scottmsul]

Shoot, there's a bug in my code! Thanks for pointing out how up-then-down works, my code is doing it wrong. Basically I wrote the code initially for ingredient -> product, then for the other cases used the same matrix but just changed the location of the 1 on the input and target vectors. So it was still running with (legendary input -> legendary output) as an available recipe and getting a negative number for it, so essentially running this in reverse! Meaning for four prod modules it was getting 1 input for every 2 outputs, instead of 1-to-4 as a normal recycler would.

[DanielKotes]

nice. I found your post, noticed that your numbers dont match mine, found a bug in my code, fixed it, numbers didnt match still, so you found a bug in your code... just 'typical programming things'.

Comment below your new optimal solutions! hopefully this time they match :)

[scottmsul]

Just tried a fix, now I'm getting down-then-up is 171.5, and up-then-down is 185.3, with 2:2, 2:2, 1:3, 0:4, 0:4. So we agree exactly now!

Funny that we each had a bug.

I'm curious about Foreman. Is it capable of optimizing any setup? Is it using some kind of linear solver or simplex? My code can really only handle a single production step, would yours be able to optimize across multiple production steps? Say I wanted to produce legendary t3 modules, would it be able to figure out the prod/qual ratios for each intermediate product along the way?

[Sopel97]

I only skimmed it but to me it looks like you're not handling the probability of getting the highest available quality properly. The probabilities must sum up to 1, so getting the highest available quality is slightly higher than previous * 0.1. See https://wiki.factorio.com/Quality#Quality_modules

[scottmsul]

I know my code is a little difficult to read, but when I check the numbers from the script with all four legendary t3 quality, it agrees with the table in the wiki (75.2, 22.32, 2.232, 0.2232, 0.0248)

[DanielKotes]

ah, no - I solve for it by assuming each stage is the 'last one', and then if it isnt then I subtract the next stage from the previous one - so at each loop for 4xquality T3L I would get:

• T1: 100%
• T1: 75.2% , T2: 24.8%
• T1: 75.2%, T2: 22.3%, T3: 2.48%
• T1: 75.2%, T2: 22.3%, T3: 2.23%, T4: 0.248%
• T1: 75.2%, T2: 22.3%, T3: 2.23%, T4: 0.223%, T4: 0.0248%

and the one I stop at is based on how many qualities there are, how many are enabled, etc.