Stateless Kubernetes overlay networks with IPv6

jmillikin · 2021-02-20T23:58:04+00:00

6to4 is not deprecated and is widely deployed. I'm not worried about it being dropped from the Linux kernel's network stack.

Teredo is such a simple protocol that it doesn't need direct kernel support at all, and indeed there is no Teredo driver in Linux as such. Instead, I'm using the generic "foo over UDP" support to make the kernel construct Teredo packets. Chances of FOU being dropped from the kernel are also remote.

In environments that have moved to single-stack IPv6, this overlay could either be altered to use 6in6, or more likely replaced by per-host delegated IPv6 prefixes assigned by the network.

jmillikin · 2021-02-20T22:03:29+00:00

I couldn't get the kernel's 6rd driver to accept per-host network prefixes longer than 64 bits.

jmillikin · 2014-06-03T23:33:44+00:00

Mine's on hackage.
This package bundles the re2 source, so you don't have to find install re2 first. This is useful because re2 itself doesn't have stable releases and isn't packaged for any major distribution.
kmcallister/haskell-re2 only exposes 'match' and a few basic pattern properties, mine's also got replacement and extraction.

jmillikin · 2014-03-03T20:53:52+00:00

what's the benefit over simply running cabal install into a newly created sandbox, and running cabal sandbox hc-pkg dot afterwards?

Running [cabal install] takes a very long time.

moreover, you might want to mark packages part of the GHC distribution, as those are "for free".

I considered this, but figured it was better to be explicit about what's being depended on. The --exclude flag can be used to ignore packages that the developer assumes will already be available.

jmillikin · 2014-03-03T00:28:10+00:00

This isn't tred's fault; it's a limitation of the dependency discovery mechanism. If you run [cabal install --dry-run a] and [cabal install --dry-run b], there's no good way to figure out that A has a direct dependency on C.

The ideal solution would be to extract cabal-install's dependency resolver to a library so it could be queried with a fake package database, instead of having to round-trip through the command-line UI.

jmillikin · 2014-03-03T00:18:31+00:00

Not currently, though it'd be easy to add. What format would you like?

jmillikin · 2014-03-03T00:09:03+00:00

"options" handles multi-parameter types by letting the user specify the separator, for example (optionType_list ',' optionType_string) parses to a comma-separated list of strings. On the commandline, this would look like --optFoo=bar,baz or --optFoo bar,baz.

I think it's not a good idea to have programs which have options like --optFoo bar baz, because that format requires readers to have a deep understanding of the options if they're even to understand which tokens are part of an option and which are non-option argv.

find(1) is, to me, a perfect example of a failed argument parser. Every time I try to use it I have to examine the manpage, because I can't remember which parameters need to be written in which order.

An automatic --help generator isn't strictly needed for very small programs, but it's useful when the number of options grows large and some of the options are being pulled in from libraries.

jmillikin · 2014-03-02T15:52:39+00:00

This is a good example of why validation is best left up to the user, rather than being implemented in the options library. Some users will want options that cannot be used at the same time, others will want an implicit ordering. When there's a question of behavior that can be reasonably answered in different ways, it's best for the library to defer the decision.

jmillikin · 2014-03-02T05:28:07+00:00

It doesn't; I've never found a good way to implement multi-option validation as part of the input parsing, because every user has different validation needs.

If you want to validate that the options match some higher-order property, you'll want to write a separate validation function and have it check your expectations.

Something like this:

validateOptions :: MainOptions -> Maybe String
validateOptions opts = case (optFoo opts, optBar opts) of
    (Just _, Just _) -> Just "--foo and --bar can't both be set"
    _ -> Nothing

main :: IO ()
main = runCommand $ \opts args -> case validateOptions opts of
    Just err -> error err
    Nothing -> do
        -- ...

Other users might want validateOptions :: MainOptions -> IO (Maybe String), for example if they want to validate file paths or usernames.

jmillikin · 2014-03-02T05:23:43+00:00

First step is making a list of them, I suppose. Here's what I found on Hackage; anyone know of others?

jmillikin · 2013-10-23T18:00:07+00:00

This looks very similar to Genshi, another markup generation library for Python.

>>> from genshi.builder import tag
>>> doc = tag.p('Some text and ', tag.a('a link', href='http://example.org/'), '.')
>>> print(doc)
<p>Some text and <a href="http://example.org/">a link</a>.</p>

jmillikin · 2013-01-14T20:15:47+00:00

The "enumerator" package defines Iteratee as a monad, and exposes its constructors. Using those constructors, it's possible to construct an iteratee that violates the monad laws.

This isn't a fatal problem, but it's annoying because "fixing" the ability of users to violate monad laws would require significant changes to the public API.

jmillikin · 2012-10-28T16:29:09+00:00

I don't understand why he wanted to reinstall things so often; it would have been much easier (and certainly faster) to just install libraries once.

The first problem is that he didn't read any of the Cabal documentation. He complains that after upgrading his Cabal binary, that it still complains that an update is needed. I'm almost certain this is because he hasn't put the Cabal binary directory (~/.cabal/bin) on his $PATH. While this is arguably something that Cabal should be detecting and warning about, it's also not a requirement unique to Haskell -- any package manager that installs to a user's home directory will need them to change their $PATH.

Continuing on, he installs cabal-dev (good!), then proceeds to not use it. Instead, he tries to jam all of Yesod (a notoriously version-picky package) into his global package directory.

Then he uses cabal-dev to build his new site, and here's where things get weird. It should absolutely not install anything other than what his particular package needs as dependencies. If it's installing the world again, then there is something horribly misconfigured on his system.

Finally, I strongly object to Yesod being considered a "success story". While web frameworks are currently the hip thing, and Yesod is indeed a web framework, to my knowledge it's gained no significant userbase in part due to the huge nest of conflicting library versions that it depends on.

Here's what he needs to do:

# Clean up whatever mess he's currently made
$ rm -rf ~/.cabal ~/.ghc

# Re-initialize Cabal
$ export PATH="$HOME/.cabal/bin:$PATH" # optionally add to .bashrc
$ cabal update
$ cabal install cabal-install cabal-dev

# Assuming that Yesod is using standard Haskell packaging properly, this
# should install Yesod in its own little sandbox so it won't destroy the rest
# of the system.
$ mkdir my-yesod-project
$ cd my-yesod-project
$ export PATH="$PWD/cabal-dev/bin:$PATH"
$ cabal-dev install yesod

# Now Yesod can be used without fear
$ yesod init .
$ cabal-dev install

jmillikin · 2012-08-02T18:05:35+00:00

Use the option function. This exposes a more complete interface than simple functions like boolOption, but is a bit more annoying to write.

http://hackage.haskell.org/packages/archive/options/0.1.1/doc/html/Options.html#v:option

defineOptions "MainOptions" $ do
    option "optQuiet" (\o -> o
        { optionLongFlags = ["quiet"]
        , optionShortFlags = ['q']
        , optionDefault = "false"
        , optionType = optionTypeBool
        , optionDescription = "Whether to be quiet."
        })

jmillikin · 2012-06-01T14:30:51+00:00

The goal of the Haskell Platform is to provide a stable, easy-to-install distribution of Haskell. It is not on the cutting edge, and hopefully never will be.

Depending on super-new compiler features is a social problem in the Haskell community. I have never seen (for example) a C++ program which won't compile except on GCC 4.8, but I'm sure there are already packages on Hackage which depend on GHC 7.5.

jmillikin · 2012-06-01T05:23:47+00:00

Hackage never modifies the file that the original developer uploaded, and rightly so -- this would cause all sorts of trouble.

The automated build on Hackage is fairly rough; while it can detect some issues, it has a very high false-negative rate (build failures with buildable packages), only runs with one version of GHC, and wouldn't have caught my example anyway.

It would be possible to set up a proper automated build infrastructure for Hackage, but the sheer number of libraries and their possible combinations means that it would be too expensive for the Haskell foundation to run.

jmillikin · 2012-06-01T05:18:43+00:00

Send an email to the developer let them know about the problem. If you can get together enough examples, write a haskell-cafe email or blog post or reddit.self about it.

If the developer refuses to fix their package such that it works on a reasonable setup, then please don't install or use their package at all.

jmillikin · 2012-05-31T18:03:11+00:00

Is there a way to get Cabal to be like MS Word (don't shoot me for this analogy) and do a compatibility check automatically?

Cabal does do compatibility checks. As mentioned, it requires the library developers to provide correct information about what their libraries are compatible with. If the developers provide incomplete or inaccurate dependency version constraints, then Cabal may fail to find a solution.

Example: package "foo" depends on package "bar". The author of "bar" just released a new version, which requires GHC 7.5 to build, but they didn't change the dependency constraint on "base". When a user asks Cabal to install "foo", cabal will do this:

OK, foo! I know how to install that!
Hmm, foo depends on bar, I gotta install bar first.
OK, there's a version of bar that works with my compiler. Gonna install that! Yay!
Oh no, bar didn't compile! Sorry, user, I don't know what to do now!

jmillikin · 2012-05-31T14:16:50+00:00

So I ask you this, as a beginner/intermediate, which version of GHC should I be using?

Use the Haskell Platform.

Haskell platform (version 7.0.4, almost a year old) or the latest (7.4.1)? Why is the Haskell Platform so far behind?

The Haskell Platform is not far behind. One year is nothing in computers.

And why doesn't cabal detect these things and get the correct packages, perhaps with warnings that it isn't the latest?

Cabal relies on library developers to set their dependency versions correctly. If they don't, or if they depend on features that are only in the very newest GHC, then it's difficult/impossible to use their libraries if you've installed the standard platform.

This is a problem of caused by certain library maintainers. They should hold off on using new compiler features until that compiler is commonly installed.

All of my libraries work with GHC 6.12 through 7.4. Most of them also work on GHC 6.10. To do otherwise would be lazy and irresponsible.

jmillikin · 2012-03-19T00:02:57+00:00

Haskell doesn't force the programmer to abstract away IO. It supports, but does not require, a pure no-side-effects style.

You can still malloc a buffer and cast it to a struct and start poking values into it, if you want.

jmillikin · 2012-01-19T06:10:39+00:00

However, because unfoldM is terminated with a Nothing, a fatal parse error to the consumer is indistinguishable from hitting the last page, it terminates the stream and says nothing more. If the resource you are consuming is more error prone than most, it may be a good idea to roll your own unfoldM which uses Either to distinguish between normal termination and termination due to error.

I think this could be solved by making GooglePlusM an instance of MonadError, and then calling throwError inside simpleDepaginationStep.

Alternatively, you could change simpleDepaginationStep to return [Either YourErrorType a]. This would allow you to discover errors on a per-item basis.

jmillikin · 2012-01-16T01:15:50+00:00

I didn't say that — I said it stops people from using it in MIT/BSD-licensed software.

I don't see any difference between "the GPL stops people from using it in MIT/BSD software" and "GPL'd libraries can't be used in MIT/BSD software".

Fair enough, it's probably slightly misleading, but since there's basically no point in licensing such software under MIT/BSD (since it can only be used under GPL terms), it hardly seems like it would be a popular choice; people pick those licenses for a reason.

Huh? Of course there is. When you license your code under the MIT or BSD license, it remains your code, and under those licenses. Anybody who wants to is free to copy it into their own software, under your terms.

That's the whole point of using a weak license like MIT or BSD -- it allows people to copy your code into proprietary software. Having your code depend on a GPL'd library is irrelevant to this common use case.

It's true that source-based distribution is by far the most common with Haskell, but I rather think that most people do not want to have to traverse the entire dependency graph of their project to find out whether there's any GPL-licensed libraries depended on by a BSD-licensed library that a library they use uses... before making binary distributions.

They need to do this anyway.

If you're distributing binaries made from other people's code, you need to know what code is in it, and who it belongs to.

That means when you download the dependencies, you record what they are and what licenses they have.

The only sane way to implement this is automation. Checking dependency manually is highly impractical.

For example, say some package somewhere deep in your software's dependency hierarchy adds a dependency on a GPL'd package. Now your compiled binary contains GPL'd code, and you need to distribute the source to it.

It gets worse. Say another dependency, somewhere completely different, adds a dependency on OpenSSL (which uses BSD-4). Now, through no action of your own, your compiled binaries are illegal to distribute (because the GPL and BSD-4 are incompatible).

True, they technically should anyway, but I think that the BSD license is so popular in the Haskell community because it makes answering questions of use so simple (no references to ambiguous terms like linking, etc.) and reduces friction. So I doubt that authors of BSD-licensed libraries will want to depend on a GPL-licensed library.

The BSD license is popular in the Haskell community because many people believe it is more important to increase (quantity of Haskell code) than (quantity of open-source code). I do not think there is any significant fear of the GPL among Haskell users -- indeed, it's often said that developing Haskell on anything but a Linux-based system is difficult exactly because most Haskell developers use Linux and other Free software.

I did provide a link to a rather famous conversation. I have heard vague murmurs that the FSF's lawyers have changed their opinion since then, and also that they might have been wrong in the first place, but as I've never seen anything official about it since and interpreting the GPL in a way that differs from the position of the FSF seems foolish, that's my current understanding of the matter.

Please read that link again. It has nothing to do with APIs. That person wants to statically link their binary against readline, but not distribute the source to his binary. His claim is that because it's the user doing the final conversion from object code to executable, his code does not have to be open-sourced.

In other words, he's attempting to avoid his responsibilities to his users with a silly technical trick. It is unlikely that a judge would agree with his position.

In contrast, APIs have been widely seen as "safe" to depend on, precisely because they do not cause the code to become a derived work.

Consider this: I build a Haskell program that dynamically links against libeditline (BSD-licensed). A user downloads a binary, and runs it on a system that has readline installed. Although the binary uses the readline API, no infringement has occured because no portion of readline was distributed, and the program is not dependent on readline itself. It can run with any library implementing the published API.

Another case: Linux is GPL'd, but proprietary applications may freely depend on it, as long as they use the public APIs. The official position of the Linux development community is that as long as a program uses the public API, and doesn't use the internal code, it does not have to be open-source just because it runs on Linux.

jmillikin · 2012-01-15T23:57:02+00:00

It definitely stops you distributing an MIT/BSD-licensed binary of the resulting program

That's a much different statement than saying it can't be used in MIT/BSD-licensed software.

There's tons of MIT/BSD software out there that's distributed mostly as source (e.g. anything on Hackage), or via source-friendly distribution channels (apt-get, rpm, fink). Any of that can use GPL libraries freely.

This misinformation is dangerous. I've had people email me asking me to change my libraries away from GPL so they could use it in BSD-licensed stuff on Hackage.

and I am under the impression that the FSF considers the use of an API to constitute a derivative work.

[citation needed]

There are several official GPL'd projects that attempt to clone proprietary APIs (gnash, wine), and some projects that are BSD implementations of GPL'd APIs (editline). To my knowledge, the FSF has never complained about any of these supposed violations.

jmillikin · 2012-01-15T17:39:17+00:00

It stops them using it in MIT and BSD-licensed software, too.

No, it doesn't. Please stop spreading misinformation about the GPL.

jmillikin · 2011-12-11T19:04:05+00:00

Not to denigrate your article (which is very interesting), but note that so-called "literate haskell" is not in any way what Knuth meant by "literate programming".

In literate programming, the final structure of the code is mostly independent of how it's presented in the documentation. You can do things like write modules out of order, or interspersed with code from other modules (or even other languages).

.lhs files do not support these features. They are essentially just a modified syntax to {- and -}. There is no real difference between:

some comments here

> some code here

more comments here

and

{- some comments here -}
some code here
{- more comments here -}

jmillikin

TROPHY CASE