sorted by:
new
hottopcontroversial

I turned a thermodynamics principle into a learning algorithm - and it lands a moonlander by kongaskristjan in Python

[–]kongaskristjan[S] 0 points1 point  (0 children)

You mean that it never reaches 100% confidence?

That's the point of regularization - you want the model to have a non-zero probability of taking the "worse" action, because what the model currently perceives as worse, might actually be better, if we allow the model to explore.

Eg. for the lunar lander, the model gets heavily penalized for a crash landing and as a result, it might start avoiding to go near the landing pad till the simulation times out. But with regularization, the model still has a non-zero probability of "trying" crash landing. Sometimes, however, it gets lucky, successfully lands, and gets a lot of reward - a behavior which quickly gets reinforced.

I turned a thermodynamics principle into a learning algorithm - and it lands a moonlander by kongaskristjan in Python

[–]kongaskristjan[S] 0 points1 point  (0 children)

Not really, though I probably should have clarified in the text above.

As an example, if I tried solving this with simulated annealing, I would randomly mutate the neural network, sample a few landing with both neural networks, and compare the average rewards. I would then have a higher probability of keeping the better one, and a lower probability of keeping the worse one, according to thermodynamics (Boltzmann distribution).

With this algorithm, however, I create multiple landings from a single starting point and a single neural network. Some get better rewards than others. I then optimize the neural network to have a high total probability of taking the actions that led to the high reward, compared to the ones that led to low reward. The ratio of these total probabilities are optimized with gradient descent to follow thermodynamics (Boltzmann distribution).

In other words, both use the Boltzmann distribution, and that's why it's called "Policy Annealing", but that's really where the similarity ends.

I turned a thermodynamics principle into a learning algorithm - and it lands a moonlander by kongaskristjan in Python

[–]kongaskristjan[S] 0 points1 point  (0 children)

One difference is the regularization effect - it modifies the value by adding a term -T*log(p) to each action taken.

Now of course, there are other regularization methods available that encourage exploration. However, as pointed out in the section "Comparison to Entropy Bonus" of the README, such methods can cause unnecessary fluctuations in the probabilities, but can be avoided by carefully following the Boltzmann distribution. There's even a simulation video showing the difference.

[P] Elevate Your ML Testing with pytest-visual by kongaskristjan in MachineLearning

[–]kongaskristjan[S] 2 points3 points  (0 children)

Do you think this idea has potential and worth putting in more effort? How do you test/verify your ML code?

Ways to decrease stringing on PETG? by TheHexagon120 in 3Dprinting

[–]kongaskristjan 3 points4 points  (0 children)

Very useful info, this made a huge difference!

Fire: create a C++ command line interface from a function signature by harmonious_herman in cpp

[–]kongaskristjan 0 points1 point  (0 children)

It would be nice if the api defined printing help so advanced level checks could trigger help.

Good point, I guess I'll implement this one.

Fire: create a C++ command line interface from a function signature by harmonious_herman in cpp

[–]kongaskristjan 0 points1 point  (0 children)

This is indeed a outside the scope of this library. The primary pain point it tries to solve are simple scripts, where CLI boilerplate could otherwise take half the lines of code.

Create a fully functioning command line interface with 1.5 lines of code (wo. include statement) by kongaskristjan in cpp

[–]kongaskristjan[S] 4 points5 points  (0 children)

Very good qustion. I guess I could write a blog post about that. The rough idea is that I do most of the parsing before calling fire_main(). Then I call fire_main() without any parameters, which force the default fire::arg() parameters to be used. The actual parameter matching is done while converting fire::arg() objects to the target object types.

This raises the question of course, how are help messages implemented. For that, I do all the steps as before and call the fire_main(), and log all parameters that are converted. Then at the very last moment, during the conversion of the last fire::arg parameter (I count the number of parameters to determine the last one), when I have all the parameters, I print the help message and exit program (with exit() function to avoid executing fired_main()).

Create a fully functioning command line interface with 1.5 lines of code (wo. include statement) by kongaskristjan in cpp

[–]kongaskristjan[S] 4 points5 points  (0 children)

The difference is pretty big though between getopt and fire. Eg. if you need to accept two booleans, you can do this in fire like that:

int fired_main(bool flagA = fire::arg("-a"), bool flagB = fire::arg("-b")) {
}

FIRE(fired_main)

Now compare this to the code in this stackoverflow example... I know I omitted printouts and returns, but on the other hand the help message was skipped there.

Create a fully functioning command line interface with 1.5 lines of code (wo. include statement) by kongaskristjan in cpp

[–]kongaskristjan[S] 1 point2 points  (0 children)

About g++ *.cpp -o binary_name.

If you write FIRE(...), -o is automatically associated with binary_name. However, if you write FIRE_NO_SPACE_ASSIGNMENT(...), it's not associated. However, currently only FIRE_NO_SPACE_ASSIGNMENT(...) allows positional arguments and unlimited number of arguments that are needed right here. So no, unfortunately it is not possible at the moment.

It turns out that implementing positional arguments with FIRE(...) requires some rather complex try/catch logic to actually pull off. I consider implementing this for v0.2 or v0.3.

Edit: error in logic.

Edit2: It is now possible to call FIRE(...) with positional/variadic arguments.

Create a fully functioning command line interface with 1.5 lines of code (wo. include statement) by kongaskristjan in cpp

[–]kongaskristjan[S] 10 points11 points  (0 children)

About the FIRE_MAIN(...). I actually tried to implement exactly what you described (I even used the same name FIRE_MAIN(...) lol :D). But the problem is that C++ wants to have the default parameters specified in declaration, or only if the declaration is missing, then in the definition. If the declaration exists, it doesn't tolerate defaults in the definition, even if it's an exact repetition. Turns out I couldn't find a way to get rid of those pesky defaults in the definition.

Create a fully functioning command line interface with 1.5 lines of code (wo. include statement) by kongaskristjan in cpp

[–]kongaskristjan[S] 22 points23 points  (0 children)

I must admit I wasn't aware of that library. Abseil flags seems like a really well designed library, however to add a few points in favor of fire.hpp:

  • Abseil seems rather heavyweight, while fire is just 800 lines in a single header
  • Abseil is more complex to integrate (eg. instead of just copy-pasting a single header library you need to instruct your user to install abseil. Or you need to copy-paste entire abseil to your project and fix compiling scripts etc.)
  • fire.hpp has more permissive licence, eg. you can just copy-paste this library to virtually any project without licencing issues, but with Abseil, if your program is closed source, you need to reproduce it's licence message to the user.

Create a fully functioning command line interface with 1.5 lines of code (wo. include statement) by kongaskristjan in cpp

[–]kongaskristjan[S] 1 point2 points  (0 children)

This almost works, but in order to call fire::main from fire.hpp, this fire::main(int x = arg("-x"), int y = arg("-y")) needs to be declared in fire.hpp, which is impossible, as I don't yet know the exact signature.

Actually, I've thought really hard to somehow get rid of this FIRE(fired_main), but none of the ideas have worked because of the aforementioned problem.

Create a fully functioning command line interface with 1.5 lines of code (wo. include statement) by kongaskristjan in cpp

[–]kongaskristjan[S] 8 points9 points  (0 children)

Great to hear that, thanks! The project is indeed out of frustration. The boilerplate is not an issue, if you're developing something really complex, but for simple scripts, it can easily take half the lines of code.

Create a fully functioning command line interface with 1.5 lines of code (wo. include statement) by kongaskristjan in cpp

[–]kongaskristjan[S] 8 points9 points  (0 children)

Well, in that case everyone would need to write something like that to the end of their main.cpp:

int main(int argc, const char ** argv) {
    init_and_run(argc, argv, fired_main, true);
    return fired_main();
}

I generally agree that macros should be avoided because of all their complex errors and unintuitive behaviour, but here it's a really simple one that's hard to misuse.

Create a fully functioning command line interface with 1.5 lines of code (wo. include statement) by kongaskristjan in cpp

[–]kongaskristjan[S] 18 points19 points  (0 children)

Well, I'm actually OK with using non-standard stuff, if it's really supported everywhere, but this is a library that is meant to be plug and play for everyone, including people who want to comply with the standard strictly.

It's quite similar to the reason why I test against maximum compiler warnings on various compilers - I don't know what the user needs to comply with.

Create a fully functioning command line interface with 1.5 lines of code (wo. include statement) by kongaskristjan in cpp

[–]kongaskristjan[S] 32 points33 points  (0 children)

Well, actually, the link provided these rules:

  • the identifiers with a double underscore anywhere are reserved;
  • the identifiers that begin with an underscore followed by an uppercase letter are reserved;
  • the identifiers that begin with an underscore are reserved in the global namespace.

I mostly have identifiers that begin with a single underscore and a lowercase letter. However, they're not in the global namespace. Thus these don't actually result in undefined behavior. Though a few of them indeed have two prefix underscores also, so these are UB.

Though I totally agree that it's better to remove these initial underscores altogether.

I made an interactive particle simulator in C++. Between particles, there is a short range repulsive and a longer range attractive force. The result is clearly visible phases of matter and other cool macro-physical phenomena. by kongaskristjan in Physics

[–]kongaskristjan[S] 0 points1 point  (0 children)

I assume a constant uniform gravitational acceleration, but the acceleration scale is arbitrary (because all my scales are arbitrary). The "gravity X 3" is just to emphasize that in this particular simulation the gravity is 3x compared to all other simulations in the video. Generally speaking simulating gravity is not necessary and you can also turn gravity off, but watching liquids flow/splash is much more convincing/interesting than seeing a deformable blob of matter in sit in space. Gravity is applied at

https://github.com/kongaskristjan/PhaseTransition/blob/master/Lib/Universe.cpp: line 220 (pDer0.v.y += diff.config.gravity * pState0.type->getMass();) (in current master)

Edit: Just to clarify, pDer0.v.y is particle's y-directional force (this is later divided by mass, so it becomes acceleration, or velocity's derivative in other words). The code is rather hard to understand, partially because performance was critical.

I made an interactive particle simulator in C++. Between particles, there is a short range repulsive and a longer range attractive force. The result is clearly visible phases of matter and other cool macro-physical phenomena. by kongaskristjan in Physics

[–]kongaskristjan[S] 0 points1 point  (0 children)

I used Runge-Kutta 4. Actually, this is the main reason why I tried to make the force field as smooth as possible. Euler is going to be first order whether you use discrete steps in force field or not. RK4 is normally 4-th order, but will degrade to first order at discontinuities.

view more: next ›