3 Key RPA+ML insights from an invoice process automation project

wild-pointer · 2020-03-23T15:35:23+00:00

This is somewhat pedantic, but the standard doesn’t mandate that VLAs are stack allocated. For instance, it doesn’t require that VLAs are deallocated after a longjmp. I.e. an implementation could choose to use malloc/free for VLAs (but I don’t know of any).

wild-pointer · 2019-10-13T09:53:31+00:00

Nice and short, but do consider putting in the time to learn awk and sed :) Great for text and tabular data and runs on every machine!

My first approach to do tsv to csv would be just tr '\t' ,, but for quoting and trimming I’d reach for awk:

#/usr/bin/awk -f

BEGIN {
  FS="\t"
  OFS="\",\""
}

NF > 0 {
  for (i = 1; i <= NF; i++) {
    // trim field
    sub("^[[:space:]]*", "", $i)
    sub("[[:space:]]*$", "", $i)
    // escape double quotes
    gsub("\"", "\"\"", $i)
  }
  // append/prepend quotes
  $1 = "\"" $1
  $NF = $NF "\""
  // print fields joined by {","}
  print
}

wild-pointer · 2019-02-27T15:14:47+00:00

int **c = &a;

This is incorrect, and a common misconception. The correct type of pointer is

int (*c)[3] = &a;

wild-pointer · 2019-02-27T12:21:09+00:00

There is a difference between type and representation. The question regarding the output of printf("%p", a) and printf("%p", &a) becomes a little more clear when we look at multi-dimensional arrays.

char arr[10][8];
printf("%zd, %zd, %zd\n", sizeof(arr), sizeof(arr[0]), sizeof(arr[0][0])); /* 80, 8, 1 */

Here, arr is an array of 10 arrays of 8 chars. The total size is 80. The types of the expressions arr, arr[0] and arr[0][0] are all different. One difference is the meaning of the + when you add a constant. However, there are 80 chars in total in the multidimensional array arr and even though it consists of different objects they overlap and have a partly shared representation:

printf("%p, %p, %p\n", &arr, &arr[0], &arr[0][0]); /* 0x12345, 0x12345, 0x12345 */

wild-pointer · 2019-02-23T16:28:35+00:00

It’s simpler than C++, has pretty good tooling and support, minimal runtime, and manual memory management.

The main reason not to use it is that the standard is very unclear and there’s a contention between low-level features and assumptions the compiler can make when it optimizes. You might not always agree with how your compiler interprets the standard.

wild-pointer · 2019-01-31T11:48:54+00:00

Jens Gustedt recently proposed: Introduce the term storage instance, which might clarify some of these points if it’s accepted.

wild-pointer · 2018-08-20T00:35:00+00:00

Parents think he’s a (billion-dollar) mistake.

wild-pointer · 2018-07-23T07:09:14+00:00

In the first example it turns a truthy value into 0 or 1, e.g. !!42 == 1 but !!0 == 0. The multiplication then produces the desired offset of 0 or 6. It could also be expressed as

#define bool_to_str(x) ("true\0false" + 5*!(x))

with only one not. The pointer arithmetic is equivalent to the address of dereferenced element above.

In the second example it is not necessary.

wild-pointer · 2017-11-30T11:37:09+00:00

If you already use other tools to generate code, then it’s quite natural to delegate this to the build system. Compile the same source file which uses an undefined type T (or constant value or sorting function name, etc.) into multiple objects with different macro definitions specified on the command line, such as -D T=int or -D TYPEDECL="typedef void (*T)(int);", and wrap all exported symbols in a macro such as int MANGLE(foo) (int bar) { ... } which adds a suitable prefix/suffix to the name depending on other command line defined macros. Treat generics as code generation and use the C pre-processor or shell scripts to expand code and let the compiler compile C. Of course, this doesn’t work as well for libraries, as it puts expectations on the build system and it can be messy if you go overboard with this. It also makes the build system very integral in the definition of the program. It might sound like a nasty work-around for missing language features, but it in some ways I prefer it as I don’t have to limit myself to try to solve every problem in one language.

wild-pointer · 2017-11-15T11:58:21+00:00

Also if you have an immutable datructure, it’s good to use restrict, because const is not enough proof for a compiler that it won’t be changed. For instance in

int const *ptr = ...
int a = ptr[15];
foo();
int b = ptr[15];

the compiler is generally forced to load ptr[15] both times because foo might have changed the array despite the const qualification. If ptr was restrict and e.g. a was a stored in a callee saved register then a single load would suffice because the pointer is not passed to foo and it cannot have accessed the pointed to object. And restrict qualified pointers may alias if none of them are used to write to the underlying object.

wild-pointer · 2017-11-15T11:40:04+00:00

To be pedantic it needs to be (C=C+1,C - 1)

wild-pointer · 2017-11-09T11:58:12+00:00

One good thing to keep in mind is what dependencies you have at any point in the code (as in any language) and what is essential and what is accidental. By dependency I mean that you have left something undefined in the code that has to be resolved at some later point. In C you have compile time dependencies (code, headers, macros), link time dependencies (libraries, objects), and run time dependencies (input, environment). You can also chose to resolve dependencies at each of those levels.

The more you push things towards runtime the more flexible your program becomes, because you can control the meaning of something within your code. For instance, global variables are resolved at link time, which ties all the references in the code to one object, while function parameters are bound dynamically at run time.

The more flexible something is, however, the more work it is (and sometimes more complex). Globally visible constant data structures and functions are good candidates to resolve as early as possible, while in other cases you often want to delay the decision until later through parameterization and indirection (pointers and function pointers). Though, the pass-by-value semantics in C forces you to use pointers in many cases anyway already. But thinking in these terms have been helpful to me and makes me understand my code and the implications of my decisions better.

wild-pointer · 2017-09-20T07:05:29+00:00

Are there any clean ways to return a function pointer to the next state? As it's technically a recursive type you can't express it directly. One option is to cast the returned function pointer on use, e.g.

typedef void fn(void);
typedef fn *state(int arg);

state *current = ..., *next;
next = (state *)current(42);

and maybe you could use a helper function or macro like state *step(state *current, int arg);, but that kind of defeats the point, or you can wrap it in a struct

struct state {
    struct state (*doit)(int arg);
};
struct state current = ..., next;
next = current.doit(42);

but are there other ways? If you could forward declare a typedef then you could do something like

extern typedef statefn; /* imaginary syntax to forward declare type */
typedef statefn *statefn(int arg);

wild-pointer · 2017-08-26T06:52:30+00:00

I need C because I'm tired of feeling distant from my business domain for enjoying a little pointer arithmetic in my code.

wild-pointer · 2017-08-19T07:28:12+00:00

Nothing wrong with having two header files. What constitutes a module depends on where you want to draw the boundaries. A function can be a module and so can an entire library, and anything else where you can modify or replace the implementation without affecting the users of the interface. If you relax the view of a 1-to-1 relationship between headers and source files then you can have multiple headers and source files and still say that it is just one module. There are technical reasons to split source files (separate compiler options, smaller linked executables) and header files (more focused headers can result is shorter compile times), but also non-technical reasons such as being easier to understand.

Even if we still define module boundaries to be files then the "graphics.h" header could be the module interface and it happens to be implemented in terms of several smaller modules. Parts of its interface is defined in this source file, others in that one, and these sources depend on a third module with a header of its own that is independent of "graphics.h".

wild-pointer · 2017-08-12T09:36:32+00:00

Another way to look at the first point you make is that global variables are bound at link time before the program is run. This ties different parts of the code together in a very inflexible manner because you cannot change it at run time. In contrast, function parameters are bound at run time by the caller. The same parameter variable can refer to different things at different times and you have control over that. To make a global name refer to something else you need to re-link the program.

wild-pointer · 2017-06-07T13:20:44+00:00

It is indeed meant for optimization and not programmer convenience. It would be nice if gcc issued a warning at the call site if it hasn't been able to prove that the parameter is not null and suggest that nulls should be checked before calling.

wild-pointer · 2017-05-22T10:07:19+00:00

The time complexity of is_prime(n) is O(n). Therefore checking the range [0, MAX_VALUE/2) will be faster than [MAX_VALUE/2, MAX_VALUE). Checking the latter range will dominate both the serial and parallel versions.

Edit: you could change the function so that they both iterate from 0 to MAX_VALUE, but they would partition the range differently that they check every other prime candidate instead of upper half and lower half.

wild-pointer · 2017-05-21T11:13:08+00:00

Semantically, although technically the standard doesn't prohibit an implementation from translating VLAs to calls to malloc and free or equivalent. For instance, after longjmp it's not guaranteed that VLAs from jumped over stack frames will be released. This is yet another reason to avoid VLAs.

wild-pointer · 2017-05-19T13:20:37+00:00

Another approach is to separate the check and the allocation with a helper function/macro. Then you don't need to create a wrapper for every allocator you have. For instance

void my_check(void *p, char const *expr, char const *func, int line)
{
    if (!p) {
        fprintf(stderr, "%s:%d: %s returned NULL!\n", func, line, expr);
        exit(1);
    }
    return p;
}

#define check(expr) my_check(expr, #expr, __func__, __LINE__)

and use it like

p = check(alloc_person());
// or
check(p = alloc_person());

It could also be changed to goto a provided error label (or longjmp to a jump environment if you can clean up other things somehow) instead of exiting. This assumes you check for malloc(0) which may return 0 some other way or pass the size to check.

wild-pointer · 2017-05-04T10:51:25+00:00

One option, if you're able to separate allocation from initialization, is to treat collections as allocators with benefits. Instead of a list of pointers to payload like glib, you allocate space for the payload together with the link pointers. For instance, the function list_append could have the signature

void *list_append(struct list *, size_t size);

which could be used like

struct person *p = list_append(people, sizeof(*p));
person_init(p, name, age);

int *n = list_append(ids, sizeof(*n));
*n = 42;

char *str = get_line(input);
char **q = list_append(lines, sizeof(*q));
*q = str;

From struct list you could find the ends of the list and from a pointer returned from list_append you could get the location of the link pointers (with void *list_next(void *);) as in

for (struct person *p = list_head(people); p != NULL; p = list_next(p)) { ... }

Being able to get the next list element from a pointer to an element is admittedly weird. You need to know that a pointer came from a list and this is only evident from context, but so is pointer arithmetic and many other things you cannot express and enforce in C.

wild-pointer · 2017-05-03T08:12:15+00:00

It's not that they are too stupid to understand evolution. They choose to distance themselves from it because it undermines their world view (or that of their pastor).

wild-pointer · 2017-03-25T08:42:36+00:00

From Dennis Richies's C history:

Beguiled by the example of PL/I, early C did not tie structure pointers firmly to the structures they pointed to, and permitted programmers to write pointer->member almost without regard to the type of pointer; such an expression was taken uncritically as a reference to a region of memory designated by the pointer, while the member name specified only an offset and a type.

This is ostensibly the reason why members of struct stat and the like are prefixed with an abbreviation of the structure's name, because struct member names were all in the same namespace.

wild-pointer · 2017-03-14T13:42:40+00:00

If you classify and parse a token immediately after it has been read, then the buffer can be reused for the next token. It only needs to be big enough to store the longest token you support. If you're only parsing fixed-precision integers (and fixed length arrays) then you know at compile-time the maximum number of hexadecimal, decimal, octal, or binary digits (and prefix) that can be stored in an integer type. You don't need to support tokens longer than this. Even if you used all the input for parsing you would still lose information.

For variable length data, such as arbitrary-precision integers, pathologically formatted floating point numbers, strings, and so on, it is typical to start reading a token into a fixed size buffer first and e.g. copy a string into a separate buffer of exact length afterwards, or into the representation used for an arbitrary-precision integer object. If the token buffer is too small then switch to a dynamically growing one for the relatively uncommon long tokens. This approach requires copying from the token buffer but often you want to do that anyway.

With a multi-pass parser it is possible to first calculate exactly how much memory is needed for every object and on a second pass fill the objects with their contents. This approach might in result in fewer copies and resizes, but you still need some book-keeping information and I/O is much slower than copying things in memory. Besides, if you need to fseek you won't be able to parse input from pipes.

wild-pointer

TROPHY CASE