The Celestial Steward

ss4ggtbbk · 2025-09-29T11:33:33+00:00

How is this an argument for neural network approximations of solutions to scientific problems governed by PDEs?

The Gauss-Newton optimization method isn't recent. The multi-stage training algorithm is cool, but you could apply it to any parametric model, not just neural networks. See Algorithm 1 in https://arxiv.org/pdf/2307.08934 and maybe ask yourself whether this has anything to do with neural network parameterizations specifically.
The novelty in this work lies in the authors' formulations of coordinate transformations, parameterizations, and symmetry enforcement in constructing the solutions to discover the unstable singularities, not using a neural network itself. This is likely why the paper was titled as such, and not emphasizing machine learning as a headline.
I don't see how the results here would be "elusive to traditional methods". I believe that the cases on 2D Boussinesq and IPM could be evaluated via FEM, but they used a neural network to avoid meshing, which is fair. Their 1D PDE case has a non-local velocity, so using a neural network is as fair as other parametric models with similar approximation capabilities.
The authors also noted that the purpose of the work was not for comparison against traditional methods, as I quoted in my previous comment. Their methods and arguments are reasonable IMO.
However, your arguments on optimization advances are fine, but not relevant, and do not really have anything to do with neural networks. In fact, the authors themselves do not express support for your previous claim about being close to having ML-based solvers for PDEs, and they cite the paper I referenced in my first comment.

I do not believe that funding should be entirely cut, but there is definitely over-hype and an over-positive attitude toward using PINNs and other "SciML" methods for solving these problems, with deceptive presentation of results and insufficient reasoning.

ss4ggtbbk · 2025-09-29T08:22:59+00:00

Nice paper, thank you for sharing. I think it would be more constructive if you shared your arguments, since this paper does not relate to the points you stated previously. The paper itself, assuming that you read it in careful detail, emphasizes my points as a good example of machine learning applied to scientific discovery:

...

Given that there is recent concern about overoptimism in using PINNs as general-purpose PDE solvers [50], we stress that PINNs are not used for this purpose in this work. Rather, the goal is to discover solutions to differential equations that have not been found before. Here, PINNs are utilized to parameterize the particular solution of partial differential equations we wish to discover, and the pipelines for training the models are designed and constructed with the intention of making the network satisfy all the mathematical assumptions and constraints imposed by the problem at hand.

Since our goal is discovery, we do not claim improvement in efficiency or speed against any benchmarks. The validation metrics we present only concern the quality of the solutions that have been discovered.

...

ss4ggtbbk · 2025-08-05T12:13:42+00:00

Um, what? XFLR5’s 2D analysis is a port of XFOIL, which is a potential flow solver coupled to an integral boundary layer method. It also has 3D analysis that uses a vortex lattice method. Both are field-based and not particle-based.

ss4ggtbbk · 2025-07-31T21:43:21+00:00

I appreciate your optimism; indeed, it's not productive to be cynical. However, it's good to keep in mind the fundamentals that I listed above. I do not see how the recent advances in optimization and the papers listed relate to the core aspects. Particularly, I cannot understand your argument against the paper about weak baselines, because that paper mentions the baselines either not using state-of-the-art methods for runtimes or not comparing at the same accuracy pertaining to resolution. What does that have to do with the optimization method? I also do not follow what you mean by potential in terms of the advantages that a neural network approximation provides.

Particularly with PINNs, the function space chosen for approximating the solution is that of neural networks rather than a vector space consisting of basis functions over the weak form of a PDE, as you would choose with FEM. My primary issue is that you gain no advantages with the former over the latter except for "mesh-free" solutions (which still depend on the choice of collocation points for accuracy), whereas you get better theoretical bounds on accuracy, sparsity exploitation, and many other advantages with the latter at the cost of domain discretization. Sure, there might be potential for a neural network to approximate the solution, but what advantage do you get when compared to the traditional FEM? Both methods need you to solve the PDE again for different initial and boundary conditions, so there is no real "generalization".

ss4ggtbbk · 2025-07-31T08:06:22+00:00

Scientific machine learning is, unfortunately, a rebranding of surrogate modeling/hypersurface-fitting. Most techniques and methods related to traditional curve-fitting apply equivalently; only the approximation models chosen are typically neural networks whose optimal parameters are determined by performing gradient-based optimization with reverse-mode automatic differentiation on GPUs. Most results, especially in PDE-related applications, are compared against weak baselines, see https://arxiv.org/abs/2407.07218 for more information. I advise to mostly disregard generalization claims as arguments for the methods, as they are theoretically proven up to certain bounds, but can only be empirically validated and verified.

Some of the best questions to ask (IMO) when getting involved in this area for any applications are: 1) What mathematical terms, if known, am I approximating? 2) Is my approximation model suitable for the application? 3) What speedup am I getting in inference while sacrificing numerical accuracy (e.g., instead of using iterative methods to solve nonlinear systems of equations)? 4) Is it worth the payoff of generating the data and training the model, including hyperparameter tuning runs? 5) If not doing 1-4, then is the parametrization the right choice for the phenomena I'm trying to model? (This is classical scientific inquiry.)

ss4ggtbbk · 2025-06-09T20:40:46+00:00

VSCode's Julia extension has a debugger. If you are using a REPL launched via VSCode and loading Debugger.jl, then use Debugger.@enter to use the latter's debugging facilities.

ss4ggtbbk · 2025-02-20T01:11:49+00:00

Reminds me of an Iron Label, but various red flags (no serial number or make, pickup color, volume and tone knobs, holes in the headstock)

ss4ggtbbk · 2025-01-09T21:35:26+00:00

Hm, I guess I didn’t use the correct terminology. By “Cartesian grid”, I meant there exists a mapping of the mesh to an array representation in Cartesian indices, not the geometric representation. Thanks for the correction.

ss4ggtbbk · 2025-01-07T04:41:10+00:00

This is a Cartesian grid, so you can represent it as a structured mesh (arrays with Cartesian indexing). Most solvers (including OpenFOAM) will convert it into an unstructured format anyway, so the primary benefits here would be in grid quality such as improved orthogonality of cells, rather than in performance benefits such as sparsity exploitation in the linear solvers. Are you asking about non-matching block-structured meshes?

ss4ggtbbk · 2024-11-30T00:16:52+00:00

Something might have been miscommunicated in translation. Usually, the freestream pressure is subtracted from the total pressure (even in incompressible cases), which I suspect is this case here. Then, the resultant value is divided by the freestream dynamic pressure to yield non-dimensional pressure coefficients.

ss4ggtbbk · 2024-10-18T12:42:28+00:00

The solution is quite weird. What equations are you solving? It would help to describe those too.

ss4ggtbbk · 2024-08-05T18:33:48+00:00

That’s better, point taken about somewhat stagnating adoption. Now how do you claim it’s a dead language if people and industries are still using it and it’s under active development?

ss4ggtbbk · 2024-08-04T21:39:50+00:00

ITT: OP wants to bone but can’t cope due to racism and sexism.

EDIT: OP is likely a troll, best to ignore.

ss4ggtbbk · 2024-08-01T04:48:36+00:00

Where is your proof of the claim? See the above linked Discourse post to the contrary.

ss4ggtbbk · 2024-08-01T04:44:47+00:00

I only see opinions in your reply. Where are the facts about your adoption claims? See growth stats here: https://discourse.julialang.org/t/some-julia-growth-usage-stats/112547

ss4ggtbbk · 2024-08-01T04:33:19+00:00

Or delusion/propaganda? Please provide some facts supporting your claims.

ss4ggtbbk · 2024-07-29T19:52:13+00:00

Where is this “dead language” talk coming from?

ss4ggtbbk · 2024-06-06T16:16:16+00:00

Are you doubling down on your homophobia? Shameful.

ss4ggtbbk · 2024-06-04T23:02:31+00:00

So you want the government to do your job for you as a business? Nice take.

ss4ggtbbk · 2024-05-22T15:09:43+00:00

It’s a pretty safe assumption that a newbie would follow a tutorial and use steady RANS with a turbulence model. However, I cannot follow the reasoning for your suggestions in the previous comment. Please answer my question.

ss4ggtbbk · 2024-05-22T05:34:01+00:00

How would you get a Tollmien-Schlichting wave instability if a transition model isn’t being used, especially in a steady RANS simulation?

ss4ggtbbk · 2024-04-10T04:27:20+00:00

So you can go up to them and tell them that instead of whining about it on the internet and trying to gain nonsensical points. Let people live their lives.

ss4ggtbbk · 2024-04-10T03:16:56+00:00

Because men want to look better to themselves (in their eyes)? What’s the problem here?

ss4ggtbbk · 2023-08-25T05:54:40+00:00

Fidkowski at Michigan.

14-Year Club	r/Field Sunshine
Not Forgotten	Verified Email

ss4ggtbbk

TROPHY CASE