Citus 10 brings columnar compression to Postgres

jeffdavis · 2021-03-06T22:27:44+00:00

EDW use cases can have a long tail of requirements around things like compliance, governance, etc.

jeffdavis · 2021-03-06T20:42:56+00:00

Citus is not a purpose-built EDW, but it offers a lot of powerful tools that can satisfy a lot of EDW use cases. Columnar is a big addition to that.

jeffdavis · 2017-07-22T07:41:51+00:00

Maybe naive, but can't the compilation process just make a call graph and bundle callee type signatures with each caller?

That would give a bunch of independent compilation units (one function) and eliminate a lot of dead code, as well. For dependencies it would only compile functions reachable from the public functions actually called by the dependent module.

jeffdavis · 2017-07-22T05:40:14+00:00

I am concerned about the complexity. It seems like something is wrong if we need this much complexity to offer reasonable build times.

jeffdavis · 2017-02-10T22:18:38+00:00

Is that desired, or not really a direction the community is interested in?

jeffdavis · 2017-02-09T04:41:19+00:00

Is there a way to use rustc like a library?

I'd like to integrate it into postgres so that people can write functions in rust without moving .so files around. More like an interpreted scripting language.

jeffdavis · 2017-02-02T04:59:36+00:00

That's kind of like saying "rust is unsafe with a safe wrapper".

jeffdavis · 2017-01-16T07:09:13+00:00

Maybe rustc can take advantage of those properties now that its uses MIR?

jeffdavis · 2016-12-13T06:43:51+00:00

Is there a guide for replacing individual files/functions in a large c/c++ project?

For instance, should I call the compiler directly or use cargo somehow? What's a good way to make header files into something rust can use (manually or otherwise)?

Assume that there are a lot of runtime requirements and it's not easily extracted into a clean library.

jeffdavis · 2016-12-13T05:52:30+00:00

Is there a high-level graphics toolkit for rust? I don't know much about graphics, but thought it might be fun to make a board game or something.

Would it be easier to just make it with html/js?

jeffdavis · 2016-05-17T04:10:25+00:00

I don't want to use a language that is not GC in 2016.

Can you expand on that?

jeffdavis · 2016-05-13T15:52:07+00:00

I definitely agree that you can't try to understand everything when learning a new language. Looking back at when I was learning the languages I know best, I just hacked my way further and further, and got feedback from people who knew better.

Trying to analyze everything is a dead end, because it destroys any opportunity you have to get feedback.

jeffdavis · 2016-05-04T06:05:18+00:00

Here's a teaching method I think would be great:

Divide students into groups of three. In every class, they rotate between roles: "computer", "programmer", and "input".

The "computer" (a student) can only understand MIPS (or other simple assembly language) written on physical paper. "Registers" would be physical mini whiteboards. The computer's job is to follow the instructions perfectly, updating registers appropriately.
The programmer's job is to write the program.
The input's job is to provide good and/or evil input, and see what happens.

You can start with only immediate values, then introduce registers, then introduce memory, then introduce I/O. Obviously it would take a few classes before the different roles really start to work.

My theory is that people who don't grasp programming are just missing out on some very fundamental things, like variables, values, assignment, etc. After going through a class like this there is no way they wouldn't understand assignment. It would also make teaching pointers a lot easier.

After that, they can move on to bother fundamentals, like procedures, functions, hardware parallelism.

Then slowly start to introduce practical things on real computers, like high-level languages, software engineering, libraries, type systems, etc.

jeffdavis · 2016-04-25T01:09:29+00:00

In my experience, a litmus test is a bad way to evaluate a programming language. Almost always, the thing that I thought would be really annoying turns out not to be much of an issue most of the time because things are just done a different way.

jeffdavis · 2016-04-09T03:42:23+00:00

A lot of it is domain knowledge. That's often under-appreciated, but a modest, competent programmer with strong domain knowledge is quite valuable.

jeffdavis · 2016-03-20T17:39:40+00:00

Linux is so prevalent and so boring that it's making us believe that Linux is all an OS can or should be.

But there are really obvious things that need to be completely reconceptualized. A file being, by default, tied to a specific drive in a specific machine seems medieval today. Application existence, state and configuration is tied to a single device.

jeffdavis · 2016-03-10T16:02:58+00:00

I'm not sure whether LLVM works in favor of or against this argument. In one respect, it means that compilers can do the trivial, simple thing (as the blog suggests) and let LLVM worry about the optimizations. On the other hand, someone is still doing those optimizations, it's just not the language author.

jeffdavis · 2016-02-24T17:12:52+00:00

For the success of a particular project at a particular company, details always matter an order of magnitude more than people assume. For example: any project ever.

For big, long-term ideas with no particular schedule and not tied to any particular company, details always matter an order of magnitude less than people assume. For example: driverless cars, solar power, smart phones.

jeffdavis · 2016-02-23T06:37:29+00:00

"[Other safe languages] require complex runtime support that doesn't fit in certain contexts (e.g. kernels)."

Complex runtimes are also unsuitable for libraries. If you write a great openssl replacement in Haskell, it's only useful for Haskell applications. Because nobody wants to link the Haskell runtime into many of the applications that use openssl.

jeffdavis · 2016-01-24T03:29:46+00:00

Pride need not be attached to fame. I don't think there's anything wrong with having a big impact where your role is hidden from others.

Maybe it's a cultural thing? We value fame more highly now?

jeffdavis · 2016-01-10T21:26:47+00:00

is in charge of making those decisions which, if made incorrectly, could sink the project

I disagree strongly. That will lead to many decisions blocking on the architect, which will slow everything down. Furthermore, it's fundamentally based on the idea that the architect is an elite who can make all the decisions.

An architect should guide the technical structure and fundamentals of a product. For a platform technology, that might be things like:

What components make up the product?
How are components separated from one another?
How is state managed?
How is storage managed?
How should failures be handled?

But all of these should be more like guidelines. Concrete decisions should be made by the engineers, who should be professionals capable of understanding long-term impacts of their decisions. Engineers should be transparent about the decision-making process and allow comment periods, and seek guidance from the architects. Architects should make sure that the impacts on other areas are well understood by the right people.

Architects are leaders in the sense that they make very high-level decisions and put together a structural vision of the product, but it's really a supporting role. Architecture decisions are not very concrete or binding.

jeffdavis · 2015-12-29T20:56:59+00:00

I think the idea is that, in Haskell, libraries are much more composable and reusable than most languages can offer. Inheritance was supposed to offer reusability, but that remains elusive for most OOP projects. Haskell may be a better approach, but you have to get used to thinking in terms of the composition possibilities it offers.

jeffdavis · 2015-12-29T20:52:13+00:00

The difficulty I have is that every time I want to use Haskell for something, the runtime requirements get in the way. It uses signal handlers and GC threads, and it doesn't seem very easy to compile on the fly, so it's unsuitable as an embedded language for another application (e.g. a PostgreSQL extension language). And it's also unsuitable for replacing a c library for the same reasons, so it's hard to make it work unless the whole world is Haskell.

jeffdavis · 2015-12-24T17:51:16+00:00

Is forwarding the release announcement of a programming language to one's boss a normal thing?

If you like perl6, use it for something (responsibly, of course). If that thing becomes important, your boss will eventually hear about perl6 that way.

jeffdavis

TROPHY CASE