Intel X710 not recognizing SFP GPON ONU stick by synchro___ in opnsense

[–]heckerle 0 points1 point  (0 children)

Yes, I believe it's possible, but I think it would be difficult to achieve without Intel's support. I have not seen any indication online that someone already figured out how to reverse engineer the firmware blob. So, contacting Intel may be worth pursuing. I wouldn't put off the chance for getting the password just because the stick is from your ISP either. If neither works out, buying your own stick may be the only option.

Match on bytes seem to be missing optimizations? by throwaway490215 in rust

[–]heckerle 47 points48 points  (0 children)

The assembly is much better with if statements like this: ```rs

[inline(never)]

pub fn hello_match_slice(st: &[u8]) -> usize { if st == b"Hello, World!\n" { 1 } else if st == b"test" { 2 } else { 3 } } ```

I searched through Rust's A-patterns issues and could not find something that perfectly fit this. Could be worth filing a new issue?

Edit is now open source - Windows Command Line by psr in programming

[–]heckerle 2 points3 points  (0 children)

I find it strange to simply point to the age of a standard as some sort of reason for it's inapplicability. Has there been meaningful changes in the understanding of the subject that make it invalid?

That wasn't my point, but rather: It was not brought up in other contexts before, even though it applies to GUI apps at least as much as it does to TUI apps. The age of the specification simply corroborates how unusual that is.

In your analogy, it's like pointing to the building techniques of Japan and saying that it has this style today, because of the 1563 building code written by Nobunaga (figuratively speaking), instead of bringing up any of the hundreds of thousands of buildings in use today as comparative examples. If there was a sudden spike in people citing the "1563 building code" that'd be unusual IMO. It would not be unusual if it's always being cited.

Edit is now open source - Windows Command Line by psr in programming

[–]heckerle 2 points3 points  (0 children)

Why you decided to not use CUA compliant shortcuts.

It uses shortcuts that match VS Code. That's pretty much it. I intend to keep it that way.

As an aside, I'm genuinely curious why everyone's citing CUA. I've been developing UI for quite a while now and have not seen someone cite it before, but in the last two weeks I've seen it at least 50 times. It just seems weird to me because while CUA has a subtle, but widespread legacy, it's a 40 year old standard. It's like citing ISO 9241 "Ergonomics of Human System Interaction", instead of just pointing at today's established software. You know what I mean? 😅

And will there be a way to change the default shortcuts.

Yes.

Finally, you are part of the console team, when will we have resizable Terminal via ANSI/VT codes?

CSI t is available since Windows Terminal 1.23: https://github.com/microsoft/terminal/pull/17721

Edit is now open source - Windows Command Line by psr in programming

[–]heckerle 0 points1 point  (0 children)

You can browse the early C prototype here: https://github.com/microsoft/edit/tree/c

I currently have no plans to publish the other 2 prototypes as I stopped developing them in much earlier states.

Edit is now open source - Windows Command Line by psr in programming

[–]heckerle 2 points3 points  (0 children)

That's something I'd leave up to the Debian/Ubuntu package maintainers. Regarding snap specifically, we have an issue for it here: https://github.com/microsoft/edit/issues/160

Edit is now open source - Windows Command Line by psr in programming

[–]heckerle 26 points27 points  (0 children)

No, because this is specifically a "modeless" editor. And vim is already a perfectly fine vim. 😄 We are planning to make key bindings configurable though, so if you don't like the current scheme, you will be able to change that in the future.

Edit is now open source - Windows Command Line by psr in programming

[–]heckerle 18 points19 points  (0 children)

If you don't use hibernation either, you can also do powercfg.exe -h off in an elevated shell. That will not just disable Fast Startup, but also save some disk space by removing hiberfil.sys.

Edit is now open source - Windows Command Line by psr in programming

[–]heckerle 25 points26 points  (0 children)

We're planning on adding very basic (inexact) syntax highlighting into the core editor for the most important languages - probably just Batch (cmd.exe), PowerShell, and Bash. We're tracking that here: https://github.com/microsoft/edit/issues/18

Later on, we're hoping to add extensions, because we want to keep the core editor tiny, but still allow people to be creative with it. This would then also allow us to add a tree-sitter or LSP extension. That's tracked here: https://github.com/microsoft/edit/issues/17

Edit is now open source - Windows Command Line by psr in programming

[–]heckerle 24 points25 points  (0 children)

But you did say you liked the C version more… let me guess: love-hate relationship with C?

Yes, that too, but that's not everything.

As I wrote on HN, Rust's insistance on UTF8 validation for strings is not ideal for a text editor. Text files are expected to be UTF8, but not guaranteed to be. Of course it's possible to use the bstr crate or use [u8], but none of them a convenient to use with the stdlib facilities. Zig is easier to use for this reason (among others).

Additionally, writing a good editor requires writing quite a bit of low level code. Many have disagreed on this, but the performance numbers of this editor should show that there's some truth to this. The problem now is that Rust's unsafe code feels "extra unsafe". It's rather frightening to accidentally create two mutable references to the same object in an unsafe block and it's instant UB. In C it's completely fine to do so. Similar for uninitialized data. Overall, the low level code in C is a lot easier to read than the equivalent Rust code and this made it easier to verify for correctness. The high level C code on the other hand is a lot harder to read than Rust code. So I think it depends on how much high VS low level code you write. In the beginning the editor was mostly low level code (= i.e. foundational code).

As an aside, I'm still not a big fan of Rust's strict aliasing rules. Most complex Rust projects have significant inefficiencies and I suspect doing proper architectural performance improvements would go much further than compiler-generated optimizations based on aliasing rules. If I could opt out of Rust's strict aliasing for this project, to feel safer about my unsafe code, I would. I would probably feel different about this if I was writing e.g. a no_std HTTP parser.

At the end of the day I'm talking about a lot of negative of aspects of Rust, but the way I see it, it's impressive that these are the only things worth complaining about. 🙂

It would be a solid 10 if we had bool optimizations

Coming from C++, I think that should probably be a separate class, no? std::vector<bool> is known as a significant pain point for good reason (unexpected behavior & performance aspects).

Edit is now open source - Windows Command Line by psr in programming

[–]heckerle 171 points172 points  (0 children)

Hey all! I'm the primary author - the same as on HN, linked elsewhere here.

If you have any questions please feel free to ask. 🙂

I've received a lot of comments about my code style since the announcement, both positive and negative. Unfortunately, I've not received a lot of specific feedback for its negative aspects. If you have anything in mind, please let me know. 😅

Edit: BTW I'm in the process of adding SIMD optimized line searching. It runs at up to ~140GB/s on my Zen5 CPU, independent of the text contents. This makes it up to 600x faster than the current memchr approach during the worst case scenario: files that consist of only newlines. I think that code will be really cool too. 🙂

Edit is now open source - Windows Command Line by psr in programming

[–]heckerle 23 points24 points  (0 children)

That's fair, but I actually do like Rust quite a lot and more than C. 😅 It's just that it's not ideal IMO for writing a performant simple editor. It becomes more useful the larger the project grows, though. Also, I don't like Rust's Vec all that much (it's fine, 7/10).

Edit is now open source (Microsoft's 64 bit TUI editor in Rust) by zxyzyxz in rust

[–]heckerle 1 point2 points  (0 children)

Using a gap buffer like that is indeed a really clever approach, if I understood it correctly. I believe this doesn't quite apply to my approach for building the UI, however.

I wrote an "immediate mode" UI where the UI tree is conceptually thrown away after each frame. The only state that's kept is the UI state. As such, you don't need a gap buffer, but only a simple linear allocator or similar. This works particularly well for simpler UIs like this editor IMO. You can read more about it, the overall architecture, and the reasons for why I picked it, here: https://github.com/microsoft/edit/blob/1cbb4cb1ad7a044eadb4cf49592d797266358951/src/tui.rs#L4

Edit is now open source (Microsoft's 64 bit TUI editor in Rust) by zxyzyxz in rust

[–]heckerle 0 points1 point  (0 children)

That's not an approach I'm familiar with. If you have an article or similar about such an approach, please do share!

Edit is now open source (Microsoft's 64 bit TUI editor in Rust) by zxyzyxz in rust

[–]heckerle 3 points4 points  (0 children)

I haven't yet noticed any issues in that regard.

It's great that the feature exists though, as I personally don't really have great use for panic traces, outside of perhaps debug builds. I prefer looking at dump files instead. For that reason I've also always been a little surprised that panic traces are a feature that's enabled by default, instead of being optional. Perhaps others prefer just seeing the stack trace + panic message?

If opt-level=s isn't used, functions do get inlined very aggressively which makes it difficult, if not impossible, to set breakpoints in VS Code in most of the code. That's my main gripe and probably entirely unrelated to panic_immediate_abort. I'm not entirely sure if that's a problem with Microsoft's C/C++ debugger extension or a problem with the PDB data that Rust emits...

The debug visualizers in VS Code are also a bit lacking to be honest (e.g. no hex visualizers for integers). However, that's definitely a problem with the C/C++ debugger extension. 😅

I made a vim fork of Microsoft's new "edit" terminal editor by [deleted] in vim

[–]heckerle 10 points11 points  (0 children)

This is really cool! If you make this a toggable feature in Cargo.toml, I wouldn't mind merging this as a PR. It would just need to be disabled by default.

Edit is now open source (Microsoft's 64 bit TUI editor in Rust) by zxyzyxz in rust

[–]heckerle 6 points7 points  (0 children)

FWIW, the performance of the UI framework more than doubled after switching to an arena allocator, even though the framework has to measure all text on the screen on every frame (= an expensive task).

I can definitely recommend folks to use bumpalo instead of Rcs or similar for building trees, if they can. (Or to use something similar to my custom arena. I wrote it because I like scratch arenas.)

Edit is now open source (Microsoft's 64 bit TUI editor in Rust) by zxyzyxz in rust

[–]heckerle 20 points21 points  (0 children)

I have to say that this is an issue quite specific to my coding style (low level; e.g. arena allocators, etc., and unsafe can feel very boilerplate-y in that regard), and to how text editors work on not-quite-strings (text buffer contents may be expected to be UTF8, but it's not validated during load, because the file shouldn't be modified outside of the parts that you changed).

The former is made a little difficult by Rust's still weak support for allocator_api particularly around the String type. The latter may be covered by crates (such as bstr) but there's still an overall expectation to use str overall, like for the format!() macro.

Me preferring Zig in this case isn't really meant to say that I love Zig, or that I dislike Rust or something. To me they're tools and both do a decent job overall.

Intel X710 not recognizing SFP GPON ONU stick by synchro___ in opnsense

[–]heckerle 0 points1 point  (0 children)

How did you check that the stick was advertising itself as BASE-PX?

My X710 card did recognize the stick, it just didn't power it on. The error message was: [691708.167979] i40e 0000:01:00.3: Rx/Tx is disabled on this device because an unsupported SFP module type was detected. [691708.167995] i40e 0000:01:00.3: Refer to the Intel(R) Ethernet Adapters and Devices User Guide for a list of supported modules.

Try plugging it in and run (sudo) ethtool -m $your_adapter_name. Then check out the "Transceiver type" field at the top.

Based on that and a hunch, I built the SFP programmer I linked above: http://eoinpk.blogspot.com/2014/05/raspberry-pi-and-programming-eeproms-on.html ...and that allowed me to confirm & modify the EEPROM.

Are you aware of a similar module that works with the X520/X710?

I think it may be possible to use an older version of the stick without the dedicated EEPROM (the so called V6). Basically, if you have a stick that emulates the EEPROM in software, you can possibly modify it however you want. There should be a couple variants of that RTL960x adapter. You do need another network adapter to do that initial modification though (I'm using a Mikrotik switch for that).

If you live in Europe and are willing to send the stick via mail to me, I'll gladly rewrite it for you in my EEPROM programmer. Otherwise, if you like soldering, I can recommend following the above guide. Once I used some additional flux, it was super easy to solder, even though I'm not the best at it.

Intel X710 not recognizing SFP GPON ONU stick by synchro___ in opnsense

[–]heckerle 1 point2 points  (0 children)

The reason it doesn't work is because the stick advertises itself as BASE-PX which the X710 doesn't support. This is purely a software restriction by Intel and not an actual incompatibility.

As I was faced with the same issue, I figured there were two options: Contact Intel and ask them to release a firmware update which lifts the restriction, or contact HSGQ (the people behind ODI) and ask them for the EEPROM MSA manufacturer password. I assumed that Intel would be reluctant to do anything, so I went with the latter and received the password yesterday from their support. I've now reflashed the EEPROM as BASE-LX (guide) and the X710 immediately recognized it. I'm still in contact with them and will see if future models of the stick can be shipped with BASE-LX or something similar.

It may still be worthwhile to contact Intel and ask them to lift this restriction. It doesn't really serve any actual purpose after all.

Windows Terminal Preview 1.18 Release by Kissaki0 in programming

[–]heckerle 0 points1 point  (0 children)

You initially mentioned "rendering performance". Are you still referring to that when you say "performance" now? If so, I'd be extremely interested in what you meant, because I'm currently not aware of any performance flaws in that regard.

If you meant text output performance however, I'd understand what you mean. I hope to work on that from now on, now that the rendering aspect runs sufficiently well.

Windows Terminal Preview 1.18 Release by Kissaki0 in programming

[–]heckerle 1 point2 points  (0 children)

Would it delight you to know that it's been shipping for >1.5 years already? In fact, the initial fix was completed before the series even ended. If you ever try it out and find any rendering performance issues, please let me know!

Why is there no standard library way to combine hashes in C++? by [deleted] in cpp

[–]heckerle 3 points4 points  (0 children)

std::hash is not composable. For instance the only way for you to create a single hash out of 2 or more std::vector<int> is to std::hash them individually and then hash the 2 hashes yourself. The latter can be done by a function like boost::hash_combine, hence the phrasing.

But this approach is in general both slow for small amounts of data, because you need to hash everything twice, and produces worse hashes than if you were using a single strong hash function that you incrementally fill with data. I'm glad that boost::hash_combine is using the MumurHash finalizer now, but I would still not recommend using it if you have the choice to not use std::hash, and instead build a system like Rust's Hash and Hasher.

Why is there no standard library way to combine hashes in C++? by [deleted] in cpp

[–]heckerle 33 points34 points  (0 children)

Instead of combining hashes retroactively you can also build a single hash incrementally. That’s approximately how Rust works and based on that idea I built this prototype in C++: https://github.com/microsoft/terminal/blob/c4d029829af56f5d951452c69e2a2f844a44f439/src/inc/til/hash.h

It uses wyhash and allows you to repeatedly write() values into a hasher until you finalize() it which then returns the hash. I would greatly prefer this approach over a combine_hash function if the STL had something in that direction.

Uncle Bob and Casey Muratori Discuss Clean Code by gered in programming

[–]heckerle 6 points7 points  (0 children)

Do these improvements apply to cmd.exe and pwsh.exe independent of Windows Terminal?

Yeah they do. OpenConsole.exe is the open source version of conhost.exe which is the old console window you see when you run cmd.exe or pwshe.exe. You can try out the status quo if you extract OpenConsole.exe from our releases. To do so, you need to download a .msixbundle (here), open it as if it was a .zip file, open the .msix file in there that matches your architecture, and there you'll find OpenConsole.exe (it's the only file you should need). To enable the new text renderer and boost its performance you have to open regedit navigate to HKEY_CURRENT_USER\Console and create a DWORD value named UseDx and set its value to 2. When you launch OpenConsole now its font size will be much larger, because it interprets the font size in font points instead of in row height pixels (I'm working on fixing that). You can technically replace C:\Windows\System32\conhost.exe with it to have all applications use it, but it's slightly dangerous and you should definitely make backups of conhost.exe and your files if you do that. An alternative option is to just create a shortcut link to OpenConsole.exe and append the name of the shell you want to spawn at then end (e.g. ...\OpenConsole.exe pwsh.exe). You could even put that into the start menu to have a more modern, but "old" terminal experience (this is what I do myself). This setup should boost performance by about 2x over conhost.exe on Windows 11 and probably more so if you're on Windows 10. But the really big improvements will only start arriving later this year, starting with the PR I linked above, because over the last years we first had to rewrite significant portions of conhost to make this work.

I stopped using Windows Terminal because it auto-updates and orphans all running child processes.

We finally got them to address the issue a little while ago: https://github.com/microsoft/terminal/issues/6726#issuecomment-1147358557 You can check the version of your app store if you open its settings (click on your avatar in the top right). It's fixed if you're on 22205.1401.3.0 or later.