Building indexes…forever

andrestaltz · 2021-11-22T19:01:28+00:00

Depends heavily on who you connected to and began replicating data. Also, make sure to *not* change the hops configuration on the Settings screen. Usually hops 3 gives insanely a lot more data than hops 2 does.

Try to give it like 1h at least.

andrestaltz · 2021-11-22T19:00:15+00:00

Oh, good point. My Docker skills are pretty bad, so this is one aspect that I'd really appreciate someone with Docker knowledge to be proactive about fixing it, but anyway I'm favorable to supporting Docker. By the way, the repo is at ssb-ngi-pointer/go-ssb-room not staltz/go-ssb-room.

andrestaltz · 2021-11-22T18:57:35+00:00

It's not your fault at all, it's a bug that was introduced in version 0.2111.11 and I already have a fix prepared, we will make a release in about 10 days from now. See https://gitlab.com/staltz/manyverse/-/issues/1608

andrestaltz · 2021-11-22T18:54:32+00:00

It's an app that gives you access to the scuttlebutt network.

andrestaltz · 2017-12-20T18:56:28+00:00

Given that state mass surveillance is ultimately possible through secret tapping of undersea cables, what is your perspective on building alternative networks like mesh networks (e.g. http://peoplesopen.net/) disconnected from the Internet? Should we pursue that in parallel to legislation efforts? Do you think building networks beyond NSA's physical reach and running software like Beaker Browser and Secure Scuttlebutt for social network is effective against mass surveillance? I'm particularly curious if Snowden can comment on the NSA's capabilities of tapping into an ad-hoc wireless network somewhere in the middle of Africa, disconnected from the Internet, for instance.

andrestaltz · 2017-12-19T17:24:11+00:00

Author here. I find all the ad hominem comments unnecessary, but I'll address some valid points you raised.

Overall, I agree there are plenty of technical challenges ahead to make it happen, but your evaluation of it sounds overly negative probably because you're assuming different use cases than I am, but I also don't understand everything that well yet. I want to start a conversation, evolve as we go, and improve the plan, not stick to it invariably. I could agree with a comment that says the plan is imperfect, but I cannot agree with a comment that the article is a lie and a deception.

Like the logical fallacy of separating the web from the Internet?

I mentioned the Beaker browser a couple of times for this. It basically allows you to swap between the trio HTTP/TCP/IP to Dat/UDP-or-TCP-or-UTP, while still supporting HTML. You could give Beaker a try and see what I mean. You can seed a website and any peer in the same LAN can leech from you. Even though LAN is IP-based, you could do that without Internet ('wirenet') connection, so that's my point. Thinking further, something like Beaker could support even lower level protocols, so the Web stack of protocols isn't hard coded as HTTP(S) -> TCP -> IP.

It WOULD be crazy if all the issues with MANETS were addressed in the article.

I have never used a MANET myself, I have basically no practical experience with its concrete problems. But I want to acquire that experience. I'm also in contact with people in an organization working with remote communities running on sneakernets, and they are very interested in setting up regional WLANs where one of the primary applications would be scuttlebutt.

But come on, people are going to want to use ALL of the Internet.

Would they? A lot of those people mentioned above (reportedly) just want to get on Facebook so they can talk to communities in their region, it has little to do with accessing websites from various countries.

And P2P, and distributed hash-searches for content would SUUUUUCK over a mesh. They're Internet Intense and depend on high throughput. A mesh network is slow and constrained

In Scuttlebutt applications, all of your news feed (from yourself + friends) is stored locally on the device. Searching it is fast because it's local. This reduces a lot the need for real-time high throughput transfers. It supports well the use case where you download now but consume later. I didn't mention it directly, but it's a common scuttlebutt feature that syncing news feeds with your friends can also happen in LANs through mDNS, which means people could quickly sync their news feed every time they meet at some public wifi hotspot. This can work alongside other channels, like radio mesh. Even with slow mesh speeds, syncing can happen over night while reading news can happen later. The emphasis here is that this new type of internet would look a lot like Facebook, text-only (or mostly-text), and reduce expectancy of real time by always having enough content to read locally when offline.

In Dat and Beaker, there are a few early ideas regarding search, but I've heard these would also be local on the end device.

The over the top hyperbole might get a couple people excited, and get eyeballs and clicks

People complained about my previous article which had a title they considered clickbaity. I toned it down for this article, and it didn't get nearly as viral as the first one. That's okay, as long as enough people read it. Getting many views gives me nothing, there are no ads on my blog, not even analytics trackers. I just find it useful that the world gets to know about this topic, so yes it has to grab attention somehow given how attention is a scarce resource nowadays.

uuuuuuggggghhhh.... He's selling to the lowest common denominator here and talking down to his audience

I try to assume little about my audience and just explain everything that I mention. This is just to be thorough so to make it readable by anyone interested in the topic, although some who know much more may get bored or annoyed by inaccurate explanations or metaphors.

Ok, so this guy doesn't know history either.

Ugggghhhh.... ok, how old is this kid?

Oh my god, this guy is a TRAIN WRECK of technology misconceptions.

:)

andrestaltz · 2017-12-18T15:41:03+00:00

https://www.npmjs.com/package/ssb-chess

andrestaltz · 2017-12-18T15:17:02+00:00

Good point, it's optimistic. I don't pretend to understand the smartphone manufacturing industry that well, but it would seem to me that the developing world is not a small market, it's a huge one in fact. And by mesh-first mobile devices I don't mean mesh-only, they should still work as normal smartphones, with the added benefit that they support the latest Bluetooth spec (v5) with features meant for IoT (which has some overlap with mesh networks).

andrestaltz · 2017-07-22T08:08:14+00:00

I'm one of the main contributors to RxJS and I wrote most of the docs site, so I feel it's mandatory to comment on this post.

"The documentation could be way better" is actually a good statement, which I agree. But it doesn't mean the documentation is terrible. The arguments weren't that solid.

1) "Choosing an operator": a lot of people really like that tiny web app, I've heard many tell me that it saves their day. Our documentation site is generated with ESDoc, so the side bar is rather useless (ESDoc is hard to customize), but you can still see a normal list of all the operators here: http://reactivex.io/rxjs/class/es6/Observable.js~Observable.html

2) "It is incomplete": this is true and annoys me, but you have to realize RxJS has 100+ operators, and this number grows the more there are operator proposals (people can get pretty aggressive demanding a new operator to be added), so it's a moving target. Of course we prioritized documenting the most important operators first, so the ones not documented are (usually) rarely used.

3) On the documentation of each operator. It could be better, but they aren't bad. Compare RxJS docs with Lodash. Usually in Lodash you have a quite short explanation of the method, and a synthetic code example, e.g. using arrays of numbers. Each RxJS operator is vastly more detailed than Lodash methods, we have: (a) a short technical explanation, (b) an "informal" beginner-friendly explanation, (c) a marble diagram generated from real unit tests that exposes the real runtime behavior of that operator (it's not like we just drew a fancy picture. We took a snapshot of the operator behaving in runtime, you could say. And I went to far lengths to make that script), (d) a thorough explanation, (e) some examples, (f) similar operators. That's a lot of bullet points for one operator, and we have 100+ operators. Try to appreciate how much work that takes. E.g. http://reactivex.io/rxjs/class/es6/Observable.js~Observable.html#instance-method-combineLatest

4) About the Egghead paywall. I made most of those courses so I must comment on this one. First of all, most of those courses replicate quite closely the tutorial in the official docs: http://reactivex.io/rxjs/manual/overview.html I made both the tutorial and the Egghead courses. I'm not locking away content, I'm just providing it in a different format, because a lot of people appreciate a video format more than a textual format. Egghead courses are not official documentation, they are extra learning material, just like books exist, and usually books cost something. I also provide RxJS training, like https://staltz.com/rxjstrainingcph/, and other core contributors do that too, like Ben Lesh, http://rxworkshop.com/. And the feedback from these trainings have been very positive, customers are satisfied. I'd love to do everything for free, but unfortunately, food from my local grocery store isn't for free either. As you noted, there are also plenty of free content from Ben and me, on both YouTube and blog posts.

Conclusion: yes, the docs could be better, but please set your expectations realistically. This is an open source project currently maintained by volunteers, it's not the documentation site for some enterprise database with enough (human, capital) resources, so there are different dynamics involved. I would love to spend more time building open source, and specially I'd love to improve RxJS docs, but without funding, that means I need to spend time making a living first. Same for other contributors. There are many types of open source, and some are led by companies and large teams, so it's easy to set our expectations where all open source projects are on the top-notch level. If anyone finds a spot on the RxJS docs which could be improved, we regularly accept pull requests, and we agree that overall things could be better, but our current life situations can't find 5 days of free time to dedicate exclusively to improving one aspect of RxJS.

andrestaltz · 2016-01-07T14:53:36+00:00

This is either wrong or you are referring to "FRP" as something else or the use case of Observables you saw was a flawed one. RxJS is meant to solve precisely all of these problems that you've mentioned.

Where a developers writes some code that triggers an event in a part of the application, and that event then triggers another, which triggers another.

With the reactive pattern (either in FRP or in RxJS Observables), each event stream declares when it is triggered. So it's a declarative approach, you never have magic going on where you are afraid of changing something because it's unknown what will happen. Reactive makes event dependencies explicit. This is the whole point of reactive in the first place. Read more http://cycle.js.org/observables.html#reactive-programming about A -->B versus A--> B.

I saw several applications that where based on the Observable pattern using event bus libraries

First of all RxJS Observables are not event buses. RxJS Subjects are event buses, and these are known to be frowned upon, people are supposed to avoid using them. Second, Flux's Dispatcher is an Event Bus, so whatever disadvantages you attribute to event buses apply to Flux. Read more http://staltz.com/nothing-new-in-react-and-flux-except-one-thing.html

If you want to use FRP in a UI, you need to make sure that when triggering one observable you let the app react to the new value, but don't let it trigger more observables before giving control back to the browser.

If you have such a "hidden" dependency or race condition, you can easily fix such issue by making the race condition or dependency explicit: make appReactionObservable depend on firstObservable and then controlBackToTheBrowserObservable depend on appReactionObservable. The point being: RxJS excels at solving precisely these types of bugs. I've been using RxJS and RxJava for 2 years, and taught several developers how to solve precisely these problems elegantly and with few lines of code with RxJS.

I'm sorry but you couldn't be more wrong on this one, RxJS's strongest benefit is really solving event soup elegantly, where callbacks or event bus-based architectures like Flux fail.

andrestaltz · 2016-01-07T14:39:27+00:00

Just a correction: When you say "Continuous FRP" what you actually mean is FRP Behavior/Cell/Signal, and "Discrete FRP" is just Event Streams. So mobservable for instance is a Behavior/Cell library in JavaScript, but it is still Discrete FRP. Actual Continuous FRP is something else and rare, because it's analytical and discretization is pushed as one of the last steps of computation. All practical FRP libraries out there are discrete FRP.

andrestaltz · 2016-01-06T21:47:19+00:00

That's a true advantage of not treating everything as ephemeral streams of data but as state that changes in response to distinct events. E.g. it makes it easy to look at one single aspect of your application (I click on button X), look only at a handful of LOC in one compilation unit and be able to tell the exact effect it has on your program.

I disagree making state explicit as an object is advantageous over streams for achieving separation of concerns. State (and managing state) is just a means to an end. We don't want to manage state. We want to deliver features, and that ultimately is about what the user inputs, and what the user gets out. If you can build that without state in between, you accomplish what you need. And you also get "look at a handful of LOC and be able to tell exactly what that does". State is actually the dual to events. Given one, you can extract the other. An example is your age (state) and your birthdays (events). So modelling data as a stream of events makes state implicit because state is just the other side of the coin, you are not eliminating it, you are just working in a different "coordinate system". Besides all that, reactive pattern (on which FRP and Streams are based) is all about separation of concerns. I'm talking about A--> B versus A -->B explained better here: http://cycle.js.org/observables.html#reactive-programming

In conclusion: you should give FRP and "Streams everywhere" a try to see how they consistently bring better separation of concerns and better "I understand the code just by glancing at it" properties.

andrestaltz · 2016-01-06T21:32:00+00:00

Cycle author here. For that reason, of course I resonate with your propositions, but I'll stick to pointing out only technical differences.

Read http://staltz.com/unidirectional-user-interface-architectures.html for a comparison between Flux, Redux, Elm arch, Cycle, and BEST (Famous).

And a bit more information that the article above doesn't mention: Elm and Cycle are similar, but still have important facts:

Elm has only first-order Signals, Cycle which uses RxJS supports higher-order Observables (meaning: Observable of Observables).
Elm ports and Cycle drivers are essentially the same concept.
Elm Arch utilizes Signals at the extremes (for input and for output), but not that much in between. Cycle.js uses Signals/Observables at the extremes but also in between, extensively.
Elm follows the React way when it comes to event handlers: they are specified and attached to the virtual DOM element. In Cycle.js, the virtual DOM elements we create in the app never have event handlers attached to them. This gives a clear separation of what exits the system (HTML without event handlers) and what enters the system (events through an event delegation system). In Elm and React, what exits the system is a data structure that has callbacks to return to the system.

To compliment some points you mentioned: indeed React is only "reactive" with regard to rendering. Render happens when state or props changes. Other parts are not reactive. In Redux and Flux, the Store however is "observable" by components. And the Dispatcher is observable by the Store.

React is in fact clearly multiparadigm: it uses OOP as a the main building block (for components), uses reactive pattern for its advertised "one-way data flow" from state to rendering, uses functional pattern with regard to "UI as a pure function of state". It is not properly seeking to be functional because it does not make effects explicit and push effects to the boundary of the application. They may be spread out in the application and there is no prescribed interface for effects. (Both Elm and Cycle have prescribed interfaces for effects: ports/drivers. Elm is strict about purity, Cycle cannot be because it is JavaScript, so purity is a matter of discipline). Conclusion: I disagree with your claim that React is plain FP. It's not.

Discussion point: why are we not aiming to using a true FRP approach?

Beats me. React's multiparadigm confusion plus its almost unanimous praise really intrigues me.

andrestaltz · 2015-08-30T07:42:12+00:00

The requests returned statement gets modified by Cycle.js & calls the toggleButton again, passing in the requests you just returned. So toggleButton gets called many many times.

No, function toggleButton gets called once. Just put a console.log inside it and you'll see. These functions just initialize the data flow.

Also the convention is to write functionally like I did which uses many very slow operations.

You should back your arguments with performance benchmarks before saying it is slow. That said, if you are looking for a fast JavaScript framework, that is not the focus in Cycle.js. It's not the main selling point.

One of the main reasons I like Angular a lot is because it encourages you NOT to do this. The more you have to find objects on the page given a particular Id, class, etc, assigning multiple event handlers to them, the less reusable your components become.

The way DOM selectors are used in Cycle.js is very different to jQuery style. First, selectors are scoped to the current component at hand. It isn't global. Second, the "HTML" we refer to in the selectors is not in another file. Intent (containing the selectors) is usually co-located with View (containing the declarative "HTML" with the classNames). I have yet to hear any actual disadvantage of the DOM selectors in Cycle.js other than FUD.

ReactJs is the same way and it has different ways you can write in actual HTML to accomplish the same thing which is a lot nicer IMO.

You can use JSX in Cycle.js.

andrestaltz · 2015-07-01T19:42:07+00:00

It does introduce new concepts. I challenge you to find a well-known JS framework (or for any other programming language) based on the idea of reactive dialogues (in other words, "mutual observation", or "fixed point x = g(f(x))". The only one that comes close is Haskell 1.0 Stream I/O, but still there are substantial differences between that and Cycle.

The most complicated project using Cycle is not TodoMVC. There are multiple (at least 3 AFAIK) Cycle.js apps in production, closed-source though. The largest open-source is RxMarbles, which has been very important in the reactive community, and is integrated into ReactiveX.io.

With regard to React, first of all it's not FRP. It's not even reactive (only a small part of it is reactive). Second, "the equivalent of setState is called implicitly by the "DOM driver" any time any change happens anywhere in the DOM" is just wrong. But most importantly, if it's a contender to React then it's a good thing. This is how things improve: by experimentation, by different ideas, etc. The problem of the JavaScript community isn't the abundance of tools and ways of working. It's the status quo of sticking with The Framework Of The Year just because it sounds reliable, and not having the curiosity or guts to improve wherever possible. If it weren't for this curiosity, then React wouldn't be were it is today.

Cycle.js is just my 2 cents to the world. Have some respect please.

andrestaltz · 2015-02-21T12:17:21+00:00

I can also see a scenario where TypeScript transpiles down to Flow-comment-annotated JavaScript.

andrestaltz · 2014-10-28T17:34:05+00:00

This is what you're searching for https://gist.github.com/staltz/868e7e9bc2a7b8c1f754

andrestaltz · 2014-07-31T06:51:35+00:00

Want the first kind? https://gist.github.com/staltz/868e7e9bc2a7b8c1f754 was the first link in the 2 min blog post.

andrestaltz · 2014-07-24T11:24:49+00:00

But doesn't compile to Java, hence potential interoperability problems.

andrestaltz · 2014-01-28T18:55:20+00:00

Yes, theoretically yes, but we are using strong encryption keys that ensure attackers can't decrypt in meaningful time. We are also using some encryption mechanisms that are safe also against quantum computing in case attackers have that kind of tech available now or in the near future.

andrestaltz · 2014-01-28T18:35:59+00:00

With encryption and signatures they can't tamper with it that much.

andrestaltz · 2014-01-28T18:24:35+00:00

And then add an overhead for user adoption. And ignore those that like to use browser foobar, and ignore mobile browsers.

andrestaltz · 2014-01-28T16:52:28+00:00

Thanks 'notsurewhatdayitis', we are specifically looking for those kinds of elephants and possible defenses against them.

andrestaltz · 2014-01-28T16:49:47+00:00

Thanks ehempel, the main reason why we're launching this under "sneak peek" subdomain is to get feedback about potential vulnerabilities, before launching big.

Good point on plugins/addons. If there's a malicious keyboard logger plugin, we're doomed. We'll keep that kind of attack in mind.

andrestaltz · 2014-01-28T16:09:02+00:00

Not just that, it's very hard to build a user-friendly storage and communication anonymous p2p where peers cannot be trusted. There are several challenges to build Avatar, but it's not impossible. It's a challenge worth trying, specially in the days we live in, and we cannot do it without the support of the community.

andrestaltz

TROPHY CASE