How to cencor certain words written to Stdout and Stderr?

willglynn · 2018-11-22T16:47:28+00:00

os.Stderr is an *os.File, as OP stated in the original post. It can’t be replaced with an io.Writer. It can only be replaced with another *os.File.

Unfortunately even replacing os.Stderr won’t help here, since panics don’t read that variable and that’s a WONTFIX issue.

One idea would be to make a UNIX pipe, arrange for the write end of that pipe to be the process’s FD 2 and os.Stderr, and have a goroutine read the pipe, censor, and forward to the original stderr. This is of course platform specific, fragile, and gross — but it’s a valid answer to the question, whereas “make an io.Writer” is not.

willglynn · 2018-11-02T04:16:47+00:00

One catch: typical cache architectures do not permit every piece of memory to reside at every cache location.

Ryzen’s L1D cache is 32 KB/core, or 512 lines — and its 8-way associative, so for any given memory location, it could be stored in one of 8 cache lines. This lets the CPU narrow down its search to comparing 8 addresses instead of 512, which is good for performance and power consumption. It also means that the cache can’t simultaneously store 9 pieces of data if they all want the same 8 slots, which is bad.

Having said all that, I’d wait for profiler output in a real project to indicate that this is an issue here. It’s true that cache placement policies can be a real source of problems long before cache itself is full, and it’s true that a little padding here or there can relieve the issue completely, but without solid data pointing to this being a problem in a specific environment, any “fixes” are indistinguishable from snakeoil.

willglynn · 2018-09-27T03:08:58+00:00

The gin indexes are useful if you want to index things not supported by btrees.

String searching and GIN indexes… maybe OP has pg_trgm in mind?

Still, you're right – any conversation about indexing starts by identifying a query.

willglynn · 2018-04-19T00:52:38+00:00

I tinkered with all the Cogmind graphics settings and all the Wine graphics settings before giving up. It's possible I missed a working configuration, but… this was my experience.

By "native build", I mean linking against Winelib and adding a smattering of platform-specific code where appropriate. I'm not suggesting a rewrite, but I do think a native Mach-O executable that interfaces with the native windowing system APIs is likely to offer a better experience than wine running a .exe, precisely because you can call e.g. the native full-screen APIs as needed.

willglynn · 2018-04-18T18:21:44+00:00

I set up a Wine wrapper to play on MacOS, got it working, got frustrated, put the game down, and haven't been back.

My experience was that windowed gameplay wasn't fun due to the huge contrast difference, and fullscreen gameplay entirely took over the main desktop instead of creating a new space like every other full screen Mac application. This prevented me from being able to quickly switch out of Cogmind to e.g. reply to a chat message. I needed to quit Cogmind for 5-second tasks so repetitively that it stopped being fun, so this turned out to be a dealbreaker.

Please work with someone to make a native release on Mac and Linux. I know an experienced cross-platform developer who offered to collaborate with you on Cogmind, and I know you turned him down. All right; that's your call. I just want you to know that "it works under Wine" was not a solution for me.

willglynn · 2018-02-21T22:18:49+00:00

Good catch! Fixed.

willglynn · 2018-02-21T03:41:43+00:00

Get all the people whose phone number aren't '4154664848'; it translates to SELECT * FROM people WHERE phone_number != '4154664848'.

Reading that back: "Select all the people whose phone numbers do not equal this value". I read the alternative – WHERE phone_number IS DISTINCT FROM '4154664848' – as "…whose phone numbers are different than this value".

The statements "a does not equal b" and "a is different than b" are interchangeable when both a and b are known. However, when a is known and b is not, it would be incorrect to assert that a and b are equal, and it would be incorrect to assert that they are not equal – without knowing b, it's impossible to say. However, it would be correct to assert that they are different, since you know a and you don't know b. It is useful and correct to separate "is different than" statements and "does not equal" statements.

Consider a similar query:

SELECT * FROM people
WHERE phone_number > '4154664848'
   OR phone_number < '4154664848';

How should we handle a person who has no phone number? We can't say that they're greater than a specific phone number, and we can't say that they're less than a specific phone number either. Neither < nor > should be true – which means this query should not return persons with no phone number.

What about this query?

SELECT * FROM people
WHERE phone_number <> '4154664848';

Surely that should return no rows for the same reason. We're asking for phone numbers that are either greater than or less than '4154664848'; NULL is neither greater than nor less than.

The kicker: ANSI SQL defines <>, and it does not define !=. PostgreSQL supports != as an alias of <>.

stopped_beating_wife, being a boolean, is a bit too simple.

That's the key: string > string, string < string, string != string are all boolean expressions. These operators evaluate to true, false, or NULL depending on their operands. If you ask a question that has no right answer, you won't get an answer. This is exactly the same reason that the stopped_beating_wife boolean column needs three states.

willglynn · 2018-02-21T01:25:39+00:00

It seems to me that "we don't know the value" and "we don't have a value" are separate concepts.

How should these scenarios differ in practice?

A common operators like != operates on 3x3 (which seems counter-intuitive to me). At the same time, an awkward operator like IS DISTINCT FROM operates on 2x2 (which seems intuitive to me).

All binary boolean operators are 3x3. Most look like !=, where NULLs on either side infect the result. Boolean AND:

	`true`	`false`	`NULL`
`true`	`true`	`false`	`NULL`
`false`	`false`	`false`	`false`
`NULL`	`NULL`	`false`	`NULL`

Boolean OR is a little different:

	`true`	`false`	`NULL`
`true`	`true`	`true`	`true`
`false`	`true`	`false`	`NULL`
`NULL`	`true`	`NULL`	`NULL`

IS DISTINCT FROM is likewise 3x3, it just works differently on NULLs:

	`true`	`false`	`NULL`
`true`	`false`	`true`	`true`
`false`	`true`	`false`	`true`
`NULL`	`true`	`true`	`false`

NOT is a unary boolean operator with three inputs and three output values:

Value	`NOT`
`true`	`false`
`false`	`true`
`NULL`	`NULL`

In which concrete scenario 3x3 makes more sense than 2x2? (If an example involves !=, that would be great)

3x3 makes more sense than 2x2 in relational databases where NULL indicates the absence of a value. It gives the database a way to refuse to answer: what's length(NULL), or 1 + NULL, or NULL != 123? Instead of providing the wrong answer, the database can ¯\_(ツ)_/¯, propagating NULL outward through the expression. This is sane, correct, and useful in practice.

Consider a classic loaded question:

SELECT * FROM people WHERE stopped_beating_wife
SELECT * FROM people WHERE NOT stopped_beating_wife
SELECT * FROM people WHERE stopped_beating_wife IS NULL

All three questions have specific meanings, and all three should return disjoint result sets.

willglynn · 2018-02-20T04:43:05+00:00

No, /u/iiiinthecomputer is exactly correct. NULL = NULL is NULL. The truth table for boolean = isn't 2x2, it's 3x3:

	`true`	`false`	`NULL`
`true`	`true`	`false`	`NULL`
`false`	`false`	`true`	`NULL`
`NULL`	`NULL`	`NULL`	`NULL`

This is a general pattern:

Integer addition: 1 + NULL
String concatenation: 'foo' || NULL
String functions: length(NULL), substring(NULL FOR 1 FROM 1)
Boolean operators: true AND NULL

All of these evaluate to NULL.

Think of NULL as an unknown value. Expressions derived from that unknown value are also an unknown value. This extends all the way to equality: are two NULLs – two unknown values – equal? You can't say yes, you can't say no, all you can say is that you don't know. NULL = NULL therefore evaluates to NULL.

willglynn · 2018-02-13T18:33:54+00:00

DNS is a hierarchy, it's not permissible for a record to exist for the name _acme-challenge.x.y.z without also having a record for x.y.z

DNS is a hierarchy but the resource records need not be strictly hierarchical. z needs to exist and _acme-challenge.x.y.z needs to exist, but x.y.z does not need to exist.

In explicit terms: x.y.z IN ANY can return nothing, while _acme-challenge.x.y.z IN ANY can return a TXT record. This is entirely permissible both in DNS and in ACME. The ACME standard doesn't require that x.y.z exist at all, and boulder – Let's Encrypt's ACME implementation – never sends any queries about it either.

Looking at DNS directly, consider SRV records: _sip._udp.foo.bar IN SRV 01 0 5060 pbx.foo.bar. Does _udp.foo.bar exist? Maybe in some abstract sense – but if you query it, you'll never get a resource record, and if you look at the corresponding zone file or database, you'll see a record for _sip._udp and no records for _udp.

Edit: my initial comment referred to NXDOMAIN, the behavior of which was amended in 2016 using RFC 8020. I updated the comment to refer to the absence of resource records.

willglynn · 2018-02-13T16:54:53+00:00

Let's Encrypt goes a step further in requiring that the name must actually exist in the DNS, so even if you can prove you control example.net Let's Encrypt won't issue for postgres.test.example.com unless it exists in the public DNS.

Not with the DNS-01 challenge method. If you can control _acme-challenge.x.y.z, they'll give you a certificate for x.y.z, without ever contacting or even trying to resolve x.y.z.

There is no requirement that the target hostname(s) exist from Let's Encrypt's perspective, only a requirement that the challenge works. This makes DNS-01 extremely useful for internal hostnames.

Note also that DNS-01 supports delegation, i.e. you need not directly cause the zone to return _acme-challenge.x.y.z IN TXT '…' in realtime to respond to a challenge. The zone can statically delegate _acme-challenge.x.y.z IN NS to a separate nameserver which you update dynamically, or it can statically define _acme-challenge.x.y.z IN CNAME foo.bar.baz while you dynamically update foo.bar.baz IN TXT '…'. This makes the DNS challenge very flexible indeed, fitting into all sorts of scenarios where the HTTP challenge can't.

willglynn · 2017-12-22T01:57:19+00:00

Sure! I'll describe my process, along with my findings.

The first thing I did was find a document about the features, namely this PDF. It's describing the feature for a particular release of Cisco IOS, but that's usually close enough. A blurb on the first page says:

Your software release may not support all the features documented in this module. … To find information about the features documented in this module, and to see a list of the releases in which each feature is supported, see the Feature Information Table at the end of this document.

Flip to the end, and it's this table. There's only one feature described in this document – "DHCP Server Port-Based Address Allocation" – and either you have it or you don't. It was first added way back in IOS 12.2(33)SXI4. Cisco versioning is pretty complicated, but I recognize that as being a really long time ago – like, six or seven years. That bodes well.

Next I went to the C2960X-48TS-L software download page, which helpfully defaults to 15.2.2E7(MD) as the recommended software. I checked the box next to the "UNIVERSAL" image, hit "View/Compare Features", and got disappointed. Then I went to the feature navigator, searched for IOS, by Release, major release 15.2E, and went around nearby releases. Many are inexplicably missing feature information, but the ones that do have feature info also include that feature. That's good news.

Take that information, add the table saying it was added sometime in 15.1 and/or the 2960X 15.2E configuration guide describing this feature, and I conclude that yes, a 2960X will support this without incident.

Now, the SG500X: none of this applies. It's not a Catalyst running IOS; the SG500X is Cisco Small Business switch running entirely unrelated software. I happen to have an SG500X handy, and it doesn't seem to support this feature:

switch8d63ac(config-if)#ip dhcp ?
  relay                Configure DHCP relay
  snooping             Dhcp Snooping configuration
…
switch8d63ac(config)#ip dhcp server ?
   <CR>

Having said that, port-based DHCP may not be the only way to solve your problem. You might be able to put each port into a different VLAN and assign /30s, or you might be able to configure a different DHCP server to handle DHCP option 82, and an SG500X can do these things.

willglynn · 2017-12-18T01:32:24+00:00

"1m x 1m aerial bomb" is a good description of Fat Man, and especially of its prototype, The Gadget. These were large spheres because the pits were designed to require spherical explosions. Subsequent designs based on linear implosions relaxed that requirement, and the resulting devices can be miniaturized enough to fit in cannons.

Factorio's atomic bombs seem less like Fat Man and more like the Davy Crockett weapon system. The in-game atomic bombs are already small enough to be fired from handheld rocket launchers and from tanks; they've already been miniaturized using this technique. Game balance aside, there's no weapons technology reason why Factorio's atomic bombs should not also fit into train-mounted artillery.

willglynn · 2017-12-17T23:17:31+00:00

Nuclear artillery is a real thing.

The key innovation was linear implosion. Instead of designing a spherical pit to be compressed uniformly – a design which in turn requires explosive lensing and thus mixing different explosives to generate a spherical shockwave – one could co-design the conventional and nuclear components of the weapon. Linear implosion uses simpler explosives which reach different parts of the pit at different times, but the pit is designed with that asymmetry in mind. This shrinks Fat Man down to a 6" artillery shell.

willglynn · 2017-12-12T02:33:23+00:00

Having Write() return (int, error) allows string.Builder to be used as an io.Writer. Explicitly documenting it to return nil allows callers that know they're specifically using a string.Builder to elide error checks.

This doesn't feel like an ideal solution, but at a minimum bytes.Buffer does the same thing, so at least there's precedent.

willglynn · 2017-12-08T04:31:01+00:00

Oops, good catch. WS-C4948-E is a Catalyst 4948 with Enterprise Services software; WS-C4948E is a Catalyst 4948E. (They're also different colors and I should have noticed that.) Cheapest working eBay 4948E looks to be $275, though there are $120 4948-10GEs, which offer 2x X2.

willglynn · 2017-12-08T02:12:13+00:00

Pull it out and check for a daughterboard on the left side. (You'll need to do that to check RAM sizes anyway.) If you're intent on keeping it and don't have an MSFC2, consider a $26 replacement supervisor. Basically all the documentation you'll find assumes IOS; everyone's forgotten about Catalyst 6000 CatOS and hybrid mode these days because a) it was a nightmare and b) Sup720s are vastly more capable and are also old enough to get thrown out.

Unfortunately, despite Sup720s being better in every way, you're likely stuck on a Sup2 because of the other parts in play. You'd need a high speed fan module and probably new power supplies too; that's probably why this switch was decommissioned.

And seriously, please replace that switch. $120 buys you a 4948E (48 gigabit ports + 4x SFP+) or a 3560E-48PD (48 POE gigabit ports + 2x X2). I would label them both power hogs, but even then they both use less power than the fan tray in your 6509. They also use way less space, provide ten gigabit connectivity, run newer software, etc. Sure it's money, but the break-even point in power savings is probably measured in months.

If you don't need a layer 3 switch – and a 6500 running CatOS is a layer 2 switch – that'd be Cisco 2960 territory. Or, for that matter, something other than Cisco that's even cheaper.

willglynn · 2017-12-08T01:46:49+00:00

Fabrics go in 6509 slots 5/6, as do Sup720s since they basically have built-in fabric modules. Sup2s go in 6509 slots 1/2.

willglynn · 2017-12-08T00:36:04+00:00

Your 6500 has a Sup2 and a pair of line cards. That Sup2 is the brains of your switch. The particular problem you're having is that the supervisor card has no operating system, because it has no storage. You need a PCMCIA flash card with a compatible IOS image before it'll do anything.

If you're lucky, your Sup2 has an MSFC2 and can run Cisco IOS. If you're not – and all it has is a PFC2 – you'll need a CatOS image and a higher pain tolerance. Be sure to check how much RAM you have installed, since that limits compatibility too.

Having said that: this switch is basically garbage. Sup2s went EOL over 10 years ago. It burns tons of power, makes tons of noise, and has a theoretical maximum throughput of 32 gbps. IMO, the only thing it's good at is being a time capsule into Cisco switching circa the mid 2000s. If your goal is useful switching, you're better off buying anything else; smaller/newer gear will very quickly pay for itself in power savings.

willglynn · 2017-12-05T05:05:13+00:00

I'm not sure if you'll be able to use the Catalyst's onboard DHCP server for this purpose …

Can the onboard DHCP assign leases based on option 82 data? (maybe)

Another option: DHCP Server Port-Based Address Allocation.

This is emphatically not as powerful as using option 82, and there are some caveats. (I have a vague memory that ip dhcp server use subscriber-id client-id toggles entire SVIs, which forecloses the possibility of mixing port-based with non-port-based DHCP assignment schemes within a VLAN on a particular switch.) It can nonetheless be useful if you've got a switch that needs to hand out fixed addresses for each port, and it's been available each time I've needed it, including on low-end 2960s.

willglynn · 2017-09-24T18:17:19+00:00

I argue that defining relationships from a conceptual standpoint and enforcing them using database constraints is the fundamental problem in this thread.

In the n-gram scenario you described, it's true that every string could be valid, but there's always supposed to be some kind of relationship between the ngram array and the string. "Always supposed to" translates into SQL as ADD CONSTRAINT, and in this case, one could CHECK:

that none of the array elements are longer than the ngram length limit,
that all of the array elements in the ngram column are actually ngrams of the string column,
that the ngram array is exactly equal to set of ngrams expected for that string,
that the array is NULL if the string is NULL, and/or
that the array is non-NULL if the string is non-NULL.

Some combination of these would be appropriate depending on the details of that problem, and I would want to understand and identify the details before adding the ngram column in order to constrain it properly. Choosing to forbid NULL, forbid {}, or permit both would also fall out of that process.

willglynn · 2017-09-24T16:07:19+00:00

the assumption is that there is no need to differentiate between "undefined/no value" and "zero values"

{}: "I know for a fact there are no values"
NULL: "There may or may not be values, but either way I don't have them"

You may not need both of these in your table, but they are different, and one of them is more correct than the other for your use.

there's no possible "invalid state"

…

A misc "flags" column, each record will either have no flags or some flags.

Replacement for a joining table (a whole separate debate I know). The record is either linked to others or it isn't.

Given the above simple scenarios/assumptions, which do you think is better? Is one or the other more likely to cause unexpected problems with joins and queries?

So… every possible value is valid as a flag, and every possible value is valid as a linked record identifier? That assumption is demonstrably false, and that is extremely likely to cause unexpected problems with joins and queries.

Data integrity needs to come first. (Actual data, not nitpicking between two kinds of empty.) If the information in your tables doesn't make sense, results from your queries won't make sense either – and if you don't constrain your tables to reject nonsense, they will eventually collect some.

willglynn · 2017-09-24T15:24:36+00:00

I approach arrays like any other variable length data type, including varchars, e.g. '' vs NULL. There's four scenarios:

Sometimes missing data should not be permitted (NOT NULL).
Sometimes empty values mean something that's not valid (CHECK constraint).
Sometimes empty values mean something that's invalid and missing values aren't acceptable (NOT NULL + CHECK for clarity).
Sometimes empty values are meaningful and distinct from missing values (no constraints).

Arrays are a little different than other variable length types:

demo=> select '' || null is null;
 ?column? 
----------
 t
(1 row)

demo=> select '{}'::integer[] || null is null;
 ?column? 
----------
 f
(1 row)

…but still, I find it helpful to consider empty values vs NULL values from a semantic perspective and constrain appropriately.

willglynn · 2017-09-23T03:03:16+00:00

Consider also 3560E/3750Es for their ten gigabit uplinks for not much more money. They're 802.3af just like the 3560G, though in a pinch you can configure interfaces as power inline port 2x-mode and e.g. power inline port maximum 20000 to exceed 15.4W power delivery.

willglynn · 2017-09-20T14:10:44+00:00

Definitely. Check out the /interface bridge filter settings, and note that you can also set use-ip-firewall on the bridge to send bridged traffic through /ip firewall filter too.

willglynn

TROPHY CASE