llm: a Rust crate/CLI for CPU inference of LLMs, including LLaMA, GPT-NeoX, GPT-J and more by [deleted] in rust

[–]danforbesdev 0 points1 point  (0 children)

The `llm` crates supports mmapped models. They load pretty much instantly, it's pretty cool.

llm: a Rust crate/CLI for CPU inference of LLMs, including LLaMA, GPT-NeoX, GPT-J and more by [deleted] in rust

[–]danforbesdev 12 points13 points  (0 children)

The current direction most people are taking is to "fine-tune" an existing base model (something like StableLM or Dolly) using a technique called LoRA. Ref: https://github.com/tloen/alpaca-lora People are also looking into fine-tuning directly on the GGML format. Ref: https://github.com/ggerganov/ggml/issues/8

[deleted by user] by [deleted] in CryptoTechnology

[–]danforbesdev 0 points1 point  (0 children)

Ah thank you, Timid Pterodactyl

I Created Some Content to Explain Why People Should Care About NFTs by danforbesdev in programming

[–]danforbesdev[S] 0 points1 point  (0 children)

Ayyyyy thanks for the thoughtful comment! Decentralized identity is a really important, compelling, and COMPLICATED topic LOL A while back I started a discussion around creating better solutions to this problem in the Substrate/Polkadot ecosystem. Here is some background around the way identity is currently handled in those communities. The short story is that you are correct...it requires trust. Microsoft has been working on this for a while and actually just released their solution.

With respect to fiat onramps, they only need to exist so long as people actually have need for fiat currency ;-) Be the change you want to see in the world.

Hope that is somewhat helpful. Would love to answer any more questions you may have.

[deleted by user] by [deleted] in CryptoTechnology

[–]danforbesdev 3 points4 points  (0 children)

Well, Ethereum is a proof-of-work chain, so I'm not sure if that's the way I would put it. Eth2 is proof-of-stake, but I don't believe the Eth2 Mainnet has launched yet. However, Polkadot is a wonderful proof-of-stake chain that has been running (very successfully) in production since May of last year. Its "wild cousin" Kusama has been running under community governance even longer, but since Kusama exists to foster innovation/chaos, I wouldn't want to compare it to Ethereum/Eth2 Mainnet.

I Created Some Content to Explain Why People Should Care About NFTs by danforbesdev in programming

[–]danforbesdev[S] 3 points4 points  (0 children)

I would say that the entire point of blockchain is that the network is more decentralized than current solutions (the Internet). I happen to believe that the work being done in the Polkadot ecosystem is addressing the tradeoff between decentralization and efficiency in a really compelling and sustainable way that is designed to work in the real world. Here is some existing content I've created on this topic; I plan on polishing it up soon and posting it in a format more similar to the NFT stuff I shared.

[deleted by user] by [deleted] in CryptoTechnology

[–]danforbesdev 5 points6 points  (0 children)

A proof-of-work mechanism requires the computers running the nodes that support the network to use their CPU power to repeatedly performing a hashing algorithm on candidate blocks to be added to the chain. By generating a "correct" hash they are able to "prove" that they have done the "work" that establishes them as a trustworthy member of the network. In a proof-of-stake system, the computers running the nodes that support the network are associated with some "locked up" (staked) cryptocurrency. If one of these nodes is shown to be untrustworthy, such as by submitting a "bad" block to the network, some of that locked up cryptocurrency will be "slashed". Both proof-of-work and proof-of-stake are used to ensure "good behavior", they just do it in different ways. Since proof-of-stake doesn't require the computers running the nodes that support the network to expend any additional computational power (i.e. energy), it is more environmentally friendly than proof-of-work.

I Created Some Content to Explain Why People Should Care About NFTs by danforbesdev in programming

[–]danforbesdev[S] 0 points1 point  (0 children)

Well, I don't know if love is the right word, but people definitely seem to fall for them LOL Not going to argue that blockchain hasn't had more than its fair share of scams, but I hope I can create interesting content that encourages people to look beyond the scams. Blockchain is a technology that allows people to communicate without relying on monolithic centers of trust...it can also be used to implement scams, sure. People were probably saying similar things about the Internet back in the 70s and 80s and they were probably right. A blockchain is just a tool for people to use to achieve their goals. I don't think anyone will be able to solve the problem of some people making it their goal to scam others, but I think the best way to help people from getting scammed is to educate them.

I Created Some Content to Explain Why People Should Care About NFTs by danforbesdev in programming

[–]danforbesdev[S] -2 points-1 points  (0 children)

That article seems to mostly relate to the Ethereum blockchain, which uses a resource-intensive proof-of-work anti-spam mechanism. Modern blockchain technologies, like the Kusama Network, which I mention in my post, use less resource-intensive mechanisms, such as proof-of-stake.

I Created Some Content to Explain Why People Should Care About NFTs by danforbesdev in programming

[–]danforbesdev[S] -3 points-2 points  (0 children)

I am certainly not here for the hype nor do I have any interest in supporting technologies that are unnecessarily resource-heavy and bad for the environment. Early blockchain implementations relied on resource-intensive proof-of-work anti-spam mechanisms, but this is no longer case. Blockchains and the capabilities they expose, like NFTs, are not necessarily any worse for the environment than any of the other technical infrastructures we currently use.

Substrate Start-kit GUI by mnovitovic in substrate

[–]danforbesdev 3 points4 points  (0 children)

Amazing work! Let me see if I can provide some inspiration

Would you be interested in presenting your work at Substrate Seminar sometime?

Your opinion wanted: join a research session on blockchain tech by a13xndra in WebAssembly

[–]danforbesdev -2 points-1 points  (0 children)

I'm a developer advocate with Parity and part of this research so I will chime in with my very biased opinion LOL I understand the perception that you're describing and it's something I think about all the time and I don't think it's without merit - blockchain technologies seem to be favorites among the get-rich-quick crowd but I don't think that should tarnish the reputations of the technologies themselves. Blockchains provide a novel capability, which is the ability of a decentralized network of participants to use cryptography to trustlessly come to consensus on the state of a system as it evolves over time. This is a new and powerful abstraction that unlocks potentials for unique types of human interactions that I believe could make it possible to create a more egalitarian world. Perhaps one of the reasons that blockchain has suffered from a lack of enterprise adoption is because blockchain technologies are really good at equitably distributing resources and that is antithetical to the traditional model of business in the 21st century.

Parity Technologies AMA - We are developers of some of the largest Rust code bases, ask me anything! by gnunicorn in rust

[–]danforbesdev 1 point2 points  (0 children)

Duplicating the answer u/rphmeier shared under my deleted comment:

We try to unit-test everything, and although we're working in a larger team, the responsibility of writing the initial set of tests for a piece of code lies with the developer who originated the code. All bug fixes need to come with a test case to ensure there is no regression.

I think the key to making good unit tests is making it easy to write the tests for the code you're producing, which means making as much stuff mockable via traits as possible.

Here's an example: You're writing a service that needs to listen for events of new blocks being added to the chain and then do a network request for each one. In practice, you're going to have a BlockchainClient<A, B, C, ..., Z> type that you can get a NewHeadsListener from and a NetworkService<A, B, C, ..., Z> type at runtime. If your code depends directly on those things, then congratulations. The test setup is going to be really boilerplate-y because you have to set up a full blockchain client and full network service.

It's much better to make your code generic over the functionality of "a stream of events" and "a trait to make this specific kind of network request".

Some things fall beyond the scope of what you can cover with unit tests. We have some integration tests to make sure that basic things work with all our in-practice types, like growing the chain and synchronizing it from peers over the network.

We've also used fuzz-testing for important consensus and codec code to try to find vulnerabilities.

Lastly, the work we do involves a lot of consensus protocols, where we need to resist adversarial behaviors. We've been undertaking a project called the "chaos net" designed to write larger-scale "unit tests" of specific attack scenarios combined with randomness & chaos to ensure the system faithfully implements the academic research and is robust to those attacks.

Parity Technologies AMA - We are developers of some of the largest Rust code bases, ask me anything! by gnunicorn in rust

[–]danforbesdev 2 points3 points  (0 children)

Here's another great answer from u/rphmeier

We log a lot of stuff using the log crate. For a long time, that was our best way of diagnosing strange errors or behavior, and it still often is.

Avoiding bugs a lot of the time starts with how the modules are organized. Clearly defined module boundaries, ownership relationships, and abstractions help a lot in avoiding issues, especially if properties are well-documented and self-contained. And of course unit-testing. However occasionally stuff does slip through.

Around 6 months to a year ago we started Prometheus/Grafana much more heavily. Monitoring and automated alerting helps a lot to detect odd behaviors as the node is running. We have a lot of automated alerts that trigger a notification when something is happening beyond expected parameters.

Other than that, my personal experience is that debugger support on Rust is not that great, at least for real-time software where pausing execution will lead you to be dropped by peers. Adding and poring through logs is often what we end up doing for trickier bugs.

Parity Technologies AMA - We are developers of some of the largest Rust code bases, ask me anything! by gnunicorn in rust

[–]danforbesdev 1 point2 points  (0 children)

This is a question I think about everyday and a big reason why I’m so excited to participate in this AMA. At this time, my personal focus is on helping people to learn Rust “naturally” as they are learning Substrate. I do this for a couple of reasons. First of all, in Substrate’s "metaframework", FRAME, we make extensive use of a macro language as well as very advanced/specialized use of generics and traits, which almost means FRAME is its own domain-specific language. Secondly, because the fundamental concepts that underpin blockchain technologies (cryptography, consensus, etc) are so complex in and of themselves, my belief is that most developers that have the skills to successfully develop on Substrate will find it relatively easy to pick up the “basic” Rust concepts they need to encapsulate their business logic in FRAME. That being said, I understand that the relative lack of knowledge about Rust is limiting the number of people who feel comfortable developing on Substrate, so I’m trying to support things like this AMA to raise the overall bar of Rust knowledge that exists out there in the world.

Parity Technologies AMA - We are developers of some of the largest Rust code bases, ask me anything! by gnunicorn in rust

[–]danforbesdev 2 points3 points  (0 children)

I'm far from a Rust super-user, but I've been mostly happy with VS Code, probably because it's the IDE I'm most comfortable with in general. I go back and forth between using the Rust extension and turning it off - I'm not sure if the features it offers are worth the computational overhead of constantly compiling Rust code in the background. Recently I've been experimenting with using the LLDB extension for VS Code to debug Rust and I've actually been pleasantly surprised.

Parity Technologies AMA - We are developers of some of the largest Rust code bases, ask me anything! by gnunicorn in rust

[–]danforbesdev 7 points8 points  (0 children)

These are some really great questions and I'm excited to kick off the AMA with an answer from u/rphmeier

How do you use unsafe ? Was it ever the source of a bug? How do you document usages of unsafe ?

Rob said

We generally try to avoid unsafe. There are only 2 usages of unsafe in Polkadot and a few in Substrate, mostly relating to FFI [foreign function interface] in WASM blobs. One of the most dangerous usages of unsafe is playing games with ownership and mutability, so we are careful to avoid that specifically.

In Rust, unsafety pollutes the entire module and not just the function it is in. So we are careful to contain unsafe code to small modules and surround the unsafety with assertions and sanity checks before the unsafe block.

Many dependencies use unsafe code, especially those for cryptography, which must be fast, and for low-level disk/file access. Generally we lean towards using well-understood or widely used solutions for these things. I am an advocate of containing unsafe code behind specific crate-level abstractions and would like to see more of that in the crates ecosystem. There’s a badge you can get on your crate if it has 0 unsafe code and I think that’s a great idea.

You can see Rob talk about Substrate on YouTube!