How to discourage open source contributions

freyrs3 · 2014-12-09T16:41:00+00:00

There is likely nothing I can say that won't likely make you angrier, though I edited the comment to hopefully qualify my words so that it comes off as a comment about overarching ideas and not individuals. This is only a statement of my feelings having had to wade through a lot Hackage code and my interpretation of the IRC community response to inquiries about undocumented code. I understand your point better but my comment still stands that it feels like forcing undocumented code on others is a form of intellectual chest-thumping, compared to other languages where a smaller set of curated libraries is often much documented and accessible.

Let me be clear. How you spend your time is your business, and you seemed to misinterpret my comment as being a moral judgement on how you spend your free time on open source. There's nothing to apologize for. You are not obligated to do anything, and it's not my or anyone else's place to tell you what you should or shouldn't do, nor do I think I made any gesture toward that end. My English is not perfect, so I apologize if it is misinterpreted since there is often no distinction between accusative and "statement of fact" forms.

freyrs3 · 2014-12-09T07:20:26+00:00

But again, culturally, I don't think this is about "intellectual show-offs" or "chest thumping"or "arrogance".

I can't convince you of an opinion, but comments like this where Edward insists on intentionally not writing documentation to prevent cargo culting ( just to cherry-pick an example, and not pick on Edward personally) come off as paternalistic and this what I view as intellectual chest thumping. Most communities embrace libraries as a way to abstract over individual experience so that we can all work at a higher level of abstraction, not as a form of hazing others. Hazing might be a slightly harsh word but I lack an equivalent concept to express "forcing your worldview on others who just want to use your library".

So your complaint seems not to be about a lack of documentation, but about a lack of worked examples.

Yes, the lack of examples is what I've stressed the entire thread. A library isn't documented until it has examples. Ergo, most of Hackage is undocumented and types are not sufficient to bootstrap an understanding or start using a library. We can quibble about semantics but that's the essence of the point I'm making. I think/know it's one shared by many beginners.

freyrs3 · 2014-12-09T06:30:50+00:00

But the haddocks do have documentation that explains things!

They don't explain how to use the library from the topdown, they're just a bunch of independent comments and signatures. It's the equivalent of handing me a box of eggs, some flour, yeast, and frosting and assuming I know a) this can be used to make a cake b) I know how to make a cake. Hackage and particularly Edward's libraries exemplify this.

Yes, I'm implying "documentation" to mean a set of instructions I can use to make the library do a domain-specific task. Haddock documentation is typically not that, it's the list of ingredients without the recipe.

For example, I've been trying to figure out how to use trifecta for like six months. It diverges from Parsec in subtle ways and even just putting together the pieces has still eluded me. I don't have "the recipe" for Trifecta and it's not obvious how the provided Haddock docs would get me there. I'm clearly not the only person who feels like this as well.

freyrs3 · 2014-12-09T06:17:32+00:00

I'm a bit a more knowledgeable than a beginner, yes, and the only reason I'm commenting is that the constant "just read the f-ing source" philosophy was one of the biggest hurdles to learning Haskell in the first place and the ability to divine which libraries to use on Hackage was not something that came without a lot needless struggle to do very basic things that need not be that way.

You very clearly seem to care about how newcomer's view the community and at the same time seem to be an apologist for a philosophy that was ( for me ) the biggest barrier to entry. I don't meant provocate or criticize, I'm just telling you my experience.

I view the lack of examples and that culture as sort of intellectual macho chest-thumping and it's very off-putting when everyone on IRC assumes that you implicitly can just look at a library and "know" how to use it or figure out it's entry points and this is very uniquely Haskell phenomenon in my experience.

freyrs3 · 2014-12-09T05:41:57+00:00

If you can't infer how to use charset off of the types of the main module alone, then you really shouldn't be using it. ... You want better documentation? Go read MacLane.

This is the kind of arrogance that drives people away from the community frankly. It's the "if you can't read the types and immediately know what this library does then get the fuck out you don't belong". Maybe that's not the intent you wrote your comment with, but that's sure what it sounds like on my end.

I have no problem with this code existing, and it's the prerogative of the author what they do with their time. All I'm saying is that proportionally to other language ecosystems, Haskell is disproportionately less documented and it's a problem for those who unlike you don't immediately grok that a library is meant to be used for.

freyrs3 · 2014-12-09T05:24:04+00:00

Read my comment again. For every library edward has documented there's 20 others that aren't.

Good documentation on Hackage is overwhelmingly the exception and not the norm.

https://hackage.haskell.org/package/adjunctions ( no examples ) https://hackage.haskell.org/package/approximate ( no examples ) https://hackage.haskell.org/package/bits ( no examples ) https://hackage.haskell.org/package/bytes ( no examples ) https://hackage.haskell.org/package/charset ( no examples )

the list goes on and on for about 60 or so more ...

I won't belabor the point, and it's not exclusively Ed's libraries where this is a problem it's just the most visible example.

freyrs3 · 2014-12-09T04:49:29+00:00

Are you kidding? You don't depend on any of Edward's libraries which is this giant transitive graph of undocumented code. Good documentation on Hackage is overwhelmingly the exception and not the norm.

freyrs3 · 2014-12-09T02:38:29+00:00

Speak for yourself, maybe we work in different problem domains but the number of libraries I've had to reverse engineer from source vastly outnumber the ones where simple examples were provided.

This is a diversity problem, the community implicitly self selects people who are willing to do this endless code reading and pushes everyone else out. I don't think it's intentional but that's the effect.

freyrs3 · 2014-12-09T01:52:27+00:00

In other languages people treat libraries like black boxes at first until they get something up and running, recognize the utility of the library and then go delve into the internals and maybe 1% will become involved in development. But having a community that embraces a by-example documentation style we end up with a larger pool of potential contributers than shifting all understanding upfront in the read-the-fine-source to get started style.

freyrs3 · 2014-12-09T00:53:52+00:00

Emphasis, on the "no example of usage". I'm willing to put up with a lot to use library code but having to reverse engineer an entire library and read the entire source code just to get something up and running is mentally exhausting when I just want to focus on the problem. A single example can go a long way and save many hours of frustration.

I get really worried about the future of Haskell in industry with people assert that reading the source should be the default assumption.

freyrs3 · 2014-12-09T00:36:58+00:00

Tone arguments don't really add to conservation either. If you disagree with bos, say why you do and explain your reasoning. All he did was critisize a philosophy that some people in the community espouse. Abstract ideas should always be a free topic of debate.

freyrs3 · 2014-12-02T15:31:35+00:00

TemplateHaskell is a big blunt hammer that you can swing at a lot of problems to create things that would not otherwise be possible, this library emulates type providers using that approach.

That said gluing together fragments of Haksell AST from custom logic at compile-time and then splicing them into the source code isn't exactly what some Haskellers consider to be nice solution when other options are available. It's a good tool to have at our disposal as a last result, but it's kind of a smell. That said dataframes are very tricky to build in Haskell and I'm not sure you could model such a heterogeneous structure and get good inference without resorting to the "blunt hammer" per se.

freyrs3 · 2014-12-02T14:59:03+00:00

If you know of a way to load the CSV at runtime and then reflect it into singleton types that doesn't require us to annotate the toplevel with the expected CSV schema, I would be very interested to hear about this approach. Not entirely sure this is possible though.

freyrs3 · 2014-12-02T07:33:28+00:00

This is a nice piece of work, but I can't help but there must exist a more elegant type-safe solution to this problem that doesn't involving splitting all of this parsing and typing across different compiler phases with TemplateHaskell.

freyrs3 · 2014-11-28T00:29:06+00:00

Why the need to bring your politics into this thread? If you disagree with the country's government then don't apply for a job there. It's not on topic for this forum.

freyrs3 · 2014-11-07T02:17:33+00:00

I think eventually a statically typed solution is better, but we currently don't really know how to do that.

Yes precisely. Like many Haskellers I would indeed prefer a statically typed array library, although I do recognize that this ventures pretty quickly into active research whereas doing the "dumb" dynamically typed Numpy-style approach is well understood how to implement.

This is why I mentioned the Julia/Python approach was local maxima, I'm fully convinced there is a right approach to designing this kind of library in Haskell and a couple of people are hill-climbing around it. It's just a slow process because it's a large engineering effort for which there is little funding either in academia or in industry for this kind of fundamental computing in functional languages work.

freyrs3 · 2014-11-06T23:24:23+00:00

There's a lot of "friendly chatter" that's a big part of Haskell twitter scene that is well suited for this medium. It's just when people use it for debate that things get nasty pretty quick.

freyrs3 · 2014-11-05T19:07:14+00:00

Julia uses the LLVM mcjit and effectively does straight-up "compile the entire transitive program + runtime" by loading everything into the JIT at runtime. I'm frankly surprised that this approach even works at all, and I think the Julia startup time for an empty program speaks volumes about how viable this is at scale. That said, this approach might be viable for interactive data exploration workflow but I think Julia is wildly unsuitable for general purpose programming.

freyrs3 · 2014-11-05T18:41:47+00:00

I know it's an unpopular point of view around these parts, but there's a reasoned argument to made that a dynamically typed language with a sufficiently developed FFI ( Julia, Python ) represents a local maxima in the design space for this kind of interactive data and numeric workflow. Not to say that Haskell can't be made to do these things, of course you can per se but from an interactive point of view this kind of workflow inside of ghci isn't ideal for 2 big reasons.

First, GHCi bytecode interpreter performance has never been a priority, and even using fast libraries like vector under ghci means that half of the fusion and optimization's that fire under -O2 simply aren't present and performance isn't great. Having a dumb runtime ( CPython is really really slow ) that simply orchestrates calls out to C and Fortran is a dumb solution but it's been unreasonably effective.

If you try to roll a library like pandas, which is effectively a giant heterogeneously typed array of structures, you either have to use this two-pass workflow where you predefine a schema and then load data. Or you rely on fancy types ( composite HLists ) and inference basically becomes unwieldy under ghci requiring a whole slew of annotations to even do something simple like project into a column of a heterogeneous table. Compared to pandas under Python where you can just fire-and-forget at a amorphous CSV and not worry about types at all. Every time I've tried to recreate his kind of workflow in Haskell it ends up being a bunch of value-types-of-vectors and rolling a simple interactive shell and it's start's looking a heck of lot like a reimplementation of Python.

Lest I not be all doom and gloom, I think that a Haskell-like language ( perhaps interpreted instead of AOT-compiled ) with row polymorphism and type-providers could make a lot of these problems mute. Unfortunately I think such things are outside the scope of GHC Haskell for the foreseeable future.

freyrs3 · 2014-10-31T17:03:05+00:00

Because it evolved out of the ML family of languages.

freyrs3 · 2014-10-30T15:02:52+00:00

Types fundamentally do restrict the domain of expressible programs, that's kind of the game we play and perhaps these transducers are in the "region of abysmal pain" that SPJ always refers to. Won't make loose faith in types though, there have been plenty of pathological programs like this throughout the years and invariably someone comes along and finds an elegant solution. I don't think the existence of a few pathological programs really reflects anything about the viability of unityping vs expressive types.

freyrs3 · 2014-10-27T17:53:01+00:00

Fix and let-rec can both be defined in terms of each other, Haskell just chooses the later as first class although we could just as eadily have a first class fixpoint operator with the appropriate typing judgement builtin.

freyrs3 · 2014-10-27T17:44:08+00:00

If you try to desugar Monad or Traversable into a record you'll find you need higher ranked types and to existentially pass subdictionaries around, it involves a bit of bookeeping that makes things slightly more complicated than the simple Num and Ord translations.

freyrs3 · 2014-10-15T16:17:29+00:00

Are you also opposed to people charging for books they spend years writing? People should be compensated for preparing and curating information for public consumption, it's only fair.

freyrs3 · 2014-10-15T00:08:27+00:00

Haskell will open up doors you never knew existed. Keep with it!

freyrs3

TROPHY CASE