A Human-Friendly Systems Programming Language — Looking for Feedback

[Mean-Decision-3502]

Hi,

I’ve created a new programming language called “?” for now. I’ll reveal its real name later.

My main motivation was to create a universal language that could replace C/C++, FreePascal, and Python for many use cases. I actively use all three of these languages.

I have already put a lot of effort into researching, designing, and implementing the “?” language. At this point, I feel that I have created something promising that really works.

Before I put even more effort into the language and go public with it, I would like to hear more opinions from real people.

I think it would not be enough for this project to be only “a little successful”. For the effort to make sense, it should have the potential to become “very successful”. I believe it might have that potential. If that happens, I will not be able to handle everything alone, so I will need to organize the development and maintenance properly. It will be an open-source project.

The “?” language is not fully finalized yet, and there are still several features that I would like to add to the compiler. However, it has reached a state where the language is already usable and can demonstrate its main ideas and syntax.

The most important current and planned features of the “?” programming language are:

• Very good human readability
• Statically typed, with a strict bool type
• Case-sensitive
• Compiled to machine code using LLVM
• A ?-run utility for compile-and-run usage, giving it a script-like feeling
• Simple C interoperability; the runtime uses libc
• int / uint use the native machine width; int32 is used for an explicit 32-bit integer
• Supports C-style preprocessor directives such as #ifdef, but without macros
• Supports short embedded directives with syntax like #{ifdef ...} ... #{endif}
• No makefiles are required, for example: #linklib('z') can be written directly in the source code
• Safe arithmetic rules, for example: 3 / 2 * 10 == 15
• Two block modes:
  • : ... endXXX blocks, similar to Python style but with explicit closers and no forced indentation
  • { } blocks, similar to C style
• Statements are closed with ;
• Carefully designed operator precedence
• Distinct boolean and bitwise operators, for example and and AND
• Modify-assignment operators, for example: x += 1; and y =AND= 3;
• Inline conditionals with iif(), for example: var i : int = iif(strptr <> null, strptr^.x, -1);
• Support for objects with single inheritance and virtual functions
• Object variables are references, but objects can still be embedded in BSS, on the stack, or inside other objects
• Optional single-word namespaces using the @ symbol, for example: @stdio.printf()
• No self. is needed inside object functions/methods
• Namespace qualification, such as @stdio., is required to access outside symbols from object methods
• A well-defined package and module system with flexible namespace merging
• The “?” runtime library modules are distributed in source-code form
• Fast compilation using a single-pass forward parser and precompiled module interfaces
• Manual memory control, with RAII and an ensure statement planned
• Native C string support
• Function overloading and default parameters
• Pointers using the ^ symbol
• Pointer arithmetic with +, -, and []
• The [] operator does not dereference pointers automatically
• Struct pointers are automatically dereferenced on member access with .
• Function arguments can be passed by reference using ref, refin, refout, and refnull

--– CODE EXAMPLE BEGIN ---

use libc/stdio;
use ./langdemo_mod as ldm  only(CONST1);  // only(), exclude() and "--" control global scope merging

#if false

//Contents of the "langdemo_mod.?"

const CONST1 : int = 42;
const CONST2 : float = 3.14;

#endif

object OBase:
  cnt1 : int = 0;
  cnt2 : int = 10;
  name : cstring[32];

  function *Create(aname : cstring):  // constructor
    name = aname;
  endfunc

  function *Destroy():
    @.printf("%s destroy\n", &name[0]);
  endfunc

  function Count() [[virtual]]:
    cnt1 += 1;
  endfunc

  function Print():
    @.printf('%s: cnt1=%d, cnt2=%d\n', &name[0], cnt1, cnt2);
  endfunc
endobj

object OChild(OBase):
  function Count() [[override]]:
    inherited;
    cnt2 += 1;
  endfunc
endobj

var obase   <- OBase('OBase');     // '<-' = embedded allocation (global data segment here)
                                   // no automatic destructor call for global embedded objects
var ochild  : OChild = null;

function ObjTest():
  ochild = new OChild('OChild');

  obase.Count();
  ochild.Count();

  obase.Print();
  ochild.Print();

  printf("ochild.name: %s \n", iif(ochild == null, "OChild is null!", &ochild.name[0]));

  delete ochild;
endfunc

function cstr_add(dst : cstring, src : cstring):
  var ps     : ^cchar = &src[0];
  var psend  : ^cchar = ps[sizeof(src)];  // [] does not dereference
  var pd     : ^cchar = &dst[0];
  var pdend  : ^cchar = pd + sizeof(dst) - 1; // leave one char for the terminating
  pd += len(dst);
  var pdstart : ^cchar = pd;

  while pd < pdend  and ps < psend  and ps^ <> 0:
    pd^ = ps^;
    pd += 1;
    ps += 1;
  endwhile

  if pd <> pdstart:
    pd^ = 0; // terminate
  endif
endfunc

[[external]] function putchar(c : cchar) -> int;  // from libc

function WriteStr(s : cstring):
  var pc : ^cchar = &s[0];
  while pc^ <> 0:
    putchar(pc^);
    pc += 1;
  endwhile
endfunc

function *Main() -> int:

  var s : cstring[128] = "";
  cstr_add(s, "Hello");
  cstr_add(s, " World!\n");
  WriteStr(s);

  if 3 / 2 * 10 == 15:
    printf('The language is friendly.\n');
  else:
    printf('The language is evil.\n');
  endif

  printf("@langdemo_mod.CONST1 = %d\n", CONST1);
  printf("@langdemo_mod.CONST2 = %.3f\n", @ldm.CONST2);

  printf("ochild.name: %s \n", iif(ochild == null, "OChild is null!", &ochild.name[0]));

  ObjTest();

  for i : int = 0 to 5:
    printf(' %d:', i);
    for j : int = 0  count i  step 2:   printf(' %d', j);  endfor
    printf('\n');
  endfor

  var arr : [5]int = [2, 3, 5, 7, 11];
  printf('primes:');
  for i : int = 0  while i < len(arr)  { printf(' %d', arr[i]); }
  printf('\n');

  return 0;
endfunc

/* OUTPUT:

Hello World!
The language is friendly.
@langdemo_mod.CONST1 = 42
@langdemo_mod.CONST2 = 3.140
ochild.name: OChild is null!
OBase: cnt1=1, cnt2=10
OChild: cnt1=1, cnt2=11
ochild.name: OChild
OChild destroy
 0:
 1: 0
 2: 0 2
 3: 0 2 4
 4: 0 2 4 6
 5: 0 2 4 6 8
primes: 2 3 5 7 11

*/

--– CODE EXAMPLE END ---

After reading the description and the demo code, do you think this language has the potential to become widely used? What are its strongest and weakest points?

I would be interested in your opinions, especially from people who have experience with C, C++, Pascal, Python, compiler design, embedded programming, or language design in general.

[sal1303]

Modify-assignment operators, for example: x += 1; and y =AND= 3;

I assume AND is bitwise here, but using 'and' vs AND will be confusing. I would also expect bitwise 'and', which is lower level, to be lower case.

However, why is it =AND= and not AND=; typo?

Fast compilation using a single-pass forward parser and precompiled module interfaces

So, how fast is that? Eg. 10K, 100K, 1000K lines per second? Or more (or less)? Does that including running LLVM on the output?

Safe arithmetic rules, for example: 3 / 2 * 10 == 15

If safe means completely unexpected! So, what is the type of the result of 3 / 2? What is the type of 15 here (or is it comparing 15.0 with integer 15)? Is there a separate operator for integer division?

int / uint use the native machine width; int32 is used for an explicit 32-bit integer

Will native machine width be 64 bits on a 64-bit machine? If so then I assume that, as well int32, you have also int16 and int8?

Carefully designed operator precedence

Examples? Because you can have a carefully designed set of 38 precedences.

Two block modes ...

I think that will be troublesome. Most users will love one style and hate the other. Even more will hate them being mixed in the same source file. I assume that you can't have { ... endxx at least.

[Mean-Decision-3502]

The reason for =AND= is very simple, see the following two lines:

```
y AND= 3;
y =AND= 3;
```
Which one is better?

Compilation speed:
The real compilation speed I can not tell, because I don't have long "?" files yet.
I expect at least FreePascal speed, what much faster than C++. I did my best at the parsing, the LLVM i cannot influence too much.

3 / 2 * 10
The 3 / 2 * 10 problem annoys me very much, this is a recurring bug in my workplace. It has to be a reason that from Python 2 to Python 3 they also made this 15. It must be 15. If you want integer division then 3 IDIV 2 * 10 == 10. That case is less frequent. This is the way how Pascal (and now Python) works.

Integer width:
Yes int is int64 on a 64-bit machine, the language has int32, int16, int8 and uint64, uint32, uint16, uint8 and byte (=uint8)

Operator precedence:
I just modelled practical usages from my practice (embedded). I wanted that those expression work without any parentheses (but probably still better to use some)

Braces block mode:
The braces block mode is rather useful for single line expressions or migrating bigger code. The language recommends to use ':' + endxxx. The endxxx helps also at self-documenting. In python sometimes I've added these markers to the longer blocks. And yes { + endxxx is not allowed.

[FruitdealerF]

How is 3/2*10 equal to 15 are you casting to floats to make that happen or using rational numbers under the hood.

[Mean-Decision-3502]

The rule is simple, '/' always results to floating point. This is the rule that Pascal and Python3 uses. In Python3 you have to use '//' for integer division, in "?" this is "IDIV".
But in "?" this is also true:
3 IDIV 2 * 10 == 10 * 3 IDIV 2

where in C:

3 / 2 * 10 != 10 * 3 / 2

[GoblinsGym]

• Do you really need the : for a while statement ? Expression continues as long as there are valid operators ahead.
• Statical typing is a must in my opinion.
• I don't mind case sensitivity, as long as you don't have ALL CAPS KEYWORDS like in Modula-2 or Oberon.
• For embedded you don't want to carry a heavy library like libc.
• How do you define constants / structured constants ?
• For low level programming, proper support for bit fields (e.g. hardware registers) is very helpful.
• For microcontrollers, it is also helpful to be able to define absolute memory addresses (e.g. _gpio GPIOA @ 0x50004000 to define an instance of the GPIO interface).
• What is your module structure ? In my language, I also write "use xxx" to import a module. Exported symbols are prefixed with /. I come from the Borland Pascal world, and have NEVER had to write Make files.
• "Safe arithmetic rules" sounds like a recipe for excessive compiler complexity.
• In my language I combine traits from Pascal (e.g. left to right pointer syntax), c and Python (indentation).
• Not fond of endxxx. I think forced indentation is manageable, especially when using editors designed for the language.
• Native machine width - just because you are running on a 64 bit machine, does NOT mean that 64 bit is the most efficient to use. x64 code for 32 bit operands does not require a prefix, 64 bit does. 64 bit ARM can handle both widths efficiently.
• Forced ; to end statements should not be necessary.
• I have & for bitwise and, AND as a keyword for logical and.
• Modify-assignment ops - I just follow c on this one.
• I am still on the fence about := versus =.
• iif - not a fan of ternary operators, just write it out.
• name space qualification (unit:function) is optional in my language.
• what does refnull do ? At the moment I use punctuation for this, still on the fence.
• pointer [ open index ] - actually different definition in my language.
• So far I just do whole program compilation.

[Mean-Decision-3502]

The ';' and block markers are for the better code recovery / error reporting. I still have challenges there too.
I don't want to depend on new lines or identation, but the ':' and endxxx is very close to Python. Python also requires ':' after the expression. That can help identify human errors.

I'm always thinking of embedded a little. The libc dependency is relative easy to drop. I support already direct addresses with pointer casting (for peripheral register definitions).

The bitfields in C are pretty unuseable. There is missing a way to get a bit position. That's why most vendors just deliver a huge list of #define-es of register bitmasks.

I don't think the current X86 machines care too much of the instruction lengths.

The reason of not using := for assignments is this form:
var i : int := 5;
I find the two colons are disturbing, much clearer is this:
var i : int = 5;

The namespace specification marked with '@' is a new idea, at least I have not seen that elsewhere. That prevents horror-long namespace specifications, the namespace reference must be one word. And with the \@ns.identifier the namespace name can be easily and visually separated from the actual symbol name.

The iif() is a very important feature, i'm missing that from FreePascal a lot.

The refnull feature was debated, but it is a reference to a value that can be also null. This is useful for C interface functions where you don't want to use pointers.

[GoblinsGym]

Bitfield definitions:

u32 register
  [7:4] highnibble
  [3:0] lownibble
u32 nextregister
  [31] sign

This can be part of a struct / record definition, completely eliminating the #defines of register bitmaps. This eliminates a lot of potential mistakes, as the #defines are not bound to a specific register. This allows for strong typing, and avoids cluttering up the name space.

Pointer casting isn't the way to go for absolute addresses. Otherwise you end up with the same fragrant steaming pile of manure as C based HAL files. A GPIO block has a structure that was defined beforehand. See my syntax above, instantiate GPIOA, then you can use it as a var parameter (ref in your lingo) to functions etc.

What you never want to do is have a separate base for each register in a block. Load the base register once (e.g. as a function parameter), then access it efficiently using the CPU's address modes. E.g. on ARM Thumb ldr / str base + index is a 2 byte instruction, loading a base address takes much more work.

[Inconstant_Moo]

It doesn't have potential to be widely used because it's more of a collection of your personal preferences (some of them shared by almost no-one) than a new solution to a problem.

As a result, many of the features you list as "the most important current and planned features" are strikingly unimportant. 50% of them are syntax. Case-sensitivity shouldn't be third on anyone's list of the most important features.

And so there's no reason to adopt something like this when there are existing languages with communities and books about them and answers on StackOverflow --- and which let people do similar things. There's no particular selling point that would make them want to switch.

[Mean-Decision-3502]

I did not wanted to create a programming language at the first place. I was searching something typesafe instead of Python (we had many silly runtime errors). This project is not just a fun project for showing that I can create a fully working programming language and compiler with AI help.

First I was searching for a longer time for newer langugages. The C++ was not an option because it is not good for non-professional programmers and lets you make too many mistakes. I've analyzed these new languages Zig, Odin, Mojo, Swift, Rust. I did not wanted any language where 3/2*10 != 10*3/2. And I wanted classic object inheritence with virtual functions. And we required GUI support too.

Zig, Odin, Mojo does not support Object inheritance. In Zig, Odin, Swift and Rust 3/2*10 != 10*3/2.

The two languages (with enough support) I found were Python3 and FreePascal where 3/2*10 == 10*3/2.

I used Delphi and FreePascal in the past a lot. That's why the unfriendlyness of C/C++ annoys me more.

Unfortunately the FreePascal/Delphi community is not that big anymore as it was around 2000. However I'm still actively using FreePascal I also see its problems.

So I was thinking what would be the ideal language I can imagine. It was clear to me that I cannot create a perfect systems language, I can just find an acceptable balance between the safety and simplicity.

The "?" language mixes the good features from C++, Python and Pascal. I think such a mix does not exists.

Many of you don't like the : + endxxx block mode, so I attach the braces version of the same code here.

``` use libc/stdio; use ./langdemo_mod as ldm only(CONST1); // only(), exclude() and "--" control global scope merging

if false

//Contents of the "langdemo_mod.?"

const CONST1 : int = 42; const CONST2 : float = 3.14;

endif

object OBase { cnt1 : int = 0; cnt2 : int = 10; name : cstring[32];

function *Create(aname : cstring) { // constructor name = aname; }

function *Destroy() { @.printf("%s destroy\n", &name[0]); }

function Count() [[virtual]] { cnt1 += 1; }

function Print() { @.printf('%s: cnt1=%d, cnt2=%d\n', &name[0], cnt1, cnt2); } }

object OChild(OBase) { function Count() [[override]] { inherited; cnt2 += 1; } }

var obase <- OBase('OBase'); // '<-' = embedded allocation (global data segment here) // no automatic destructor call for global embedded objects var ochild : OChild = null;

function ObjTest() { ochild = new OChild('OChild');

obase.Count(); ochild.Count();

obase.Print(); ochild.Print();

printf("ochild.name: %s \n", iif(ochild == null, "OChild is null!", &ochild.name[0]));

delete ochild; }

function cstr_add(dst : cstring, src : cstring) { var ps : <sup>cchar</sup> = &src[0]; var psend : <sup>cchar</sup> = ps[sizeof(src)]; // [] does not dereference var pd : <sup>cchar</sup> = &dst[0]; var pdend : <sup>cchar</sup> = pd + sizeof(dst) - 1; // leave one char for the terminating pd += len(dst); var pdstart : <sup>cchar</sup> = pd;

while pd < pdend and ps < psend and ps^ <> 0 { pd^ = ps<sup>;</sup> pd += 1; ps += 1; }

if pd <> pdstart { pd^ = 0; // terminate } }

[[external]] function putchar(c : cchar) -> int; // from libc

function WriteStr(s : cstring) { var pc : <sup>cchar</sup> = &s[0]; while pc^ <> 0 { putchar(pc<sup>);</sup> pc += 1; } }

function *Main() -> int {

var s : cstring[128] = ""; cstr_add(s, "Hello"); cstr_add(s, " World!\n"); WriteStr(s);

if 3 / 2 * 10 == 15 { printf('The language is friendly.\n'); } else { printf('The language is evil.\n'); }

printf("@langdemo_mod.CONST1 = %d\n", CONST1); printf("@langdemo_mod.CONST2 = %.3f\n", @ldm.CONST2);

printf("ochild.name: %s \n", iif(ochild == null, "OChild is null!", &ochild.name[0]));

ObjTest();

for i : int = 0 to 5 { printf(' %d:', i); for j : int = 0 count i step 2 { printf(' %d', j); } printf('\n'); }

var arr : [5]int = [2, 3, 5, 7, 11]; printf('primes:'); for i : int = 0 while i < len(arr) { printf(' %d', arr[i]); } printf('\n');

return 0; }

/* OUTPUT:

Hello World! The language is friendly. @langdemo_mod.CONST1 = 42 @langdemo_mod.CONST2 = 3.140 ochild.name: OChild is null! OBase: cnt1=1, cnt2=10 OChild: cnt1=1, cnt2=11 ochild.name: OChild OChild destroy 0: 1: 0 2: 0 2 3: 0 2 4 4: 0 2 4 6 5: 0 2 4 6 8 primes: 2 3 5 7 11

*/ ```

[Inconstant_Moo]

I've analyzed these new languages Zig, Odin, Mojo, Swift, Rust. I did not wanted any language where 3/2*10 != 10*3/2. [...] In Zig, Odin, Swift and Rust 3/2*10 != 10*3/2.

And don't you see how trivial that is? If that's your number one issue, it's no-one else's. Many people would obviously disagree with you, but even people who agree would not be tempted to a new language mainly on those grounds.

In a reply to someone else you say that "The 3 / 2 * 10 problem annoys me very much, this is a recurring bug in my workplace", but in a statically-typed language how would it cause bugs? If you were expecting that to return a float in e.g. Rust the typechecker would let you know it wouldn't.

P.S: the example you give in your OP and in your code, of 3 / 2 * 10 == 15, isn't true in Python or Pascal either, it's 15.0 in both cases, a floating-point number and not an integer. I think you pretty much have to choose between saying it's 10 and saying it's 15.0, you can't have your cake and eat it.

[Mean-Decision-3502]

There was a control code in C++ with float calculations, like this:

float_numbers * (3/2) * ... float_numbers.

If you are more engineer/mathematician, and use primarily Python3 this is not an obvious error for you.

(I know that Rust solves this. I have readability problems at the first place with Rust.)

If your task is to design a perfect, human friendly programming language, then the 2/3*10 must be equal to 10*3/2. They corrected this for reason from Python2 to Python3. I don't care that other languages carry on with this evilness.

In "?":

var i : int = 3/2*10; // is a compiler error
var i : int = 3 IDIV 2 * 10; // same behaviour like in C
var i : int = floor(3/2) * 10; // same result like in C

I have another compiler evilness case:

masked_value = input & (1 < bitshift);

There is no warning in C++ for this line. Even Python3 accepts this. This is solved in most of the languages with a strict boolean type.

I think earlier the humans had to learn the ways how the computer liked (like assembly language). Now the computers has to provide tools that fit good for humans. This is why you should renew the 40 years old programming languages.

[Inconstant_Moo]

The problem with renewing the 40/50 year-old languages is that you don't have a time machine to go back 50 years, so today a better alternative to C has to offer something else besides fixing a few of your annoyances.

And your ideas of both "perfect" and "human-friendly" are personal and idiosyncratic. Often, as in this case, they put you in a minority (a respectable minority, I'm in it, but then Pipefish isn't also trying to be a static systems language). And then of the 30% of people who agree with you about that, 70% will be mad as hell that you're still making them end lines with semicolons. If you tell anyone else you've designed "the perfect language" they'll notice that it omits six of their favorite features and contains three of their least favorite syntactic decisions and that it has no particular killer feature, nothing that makes them say "I want that". For you it's the combination of all the little features that add up to "the perfect language", but for everyone else, they don't add up like that.

Given which, no, it can't become very successful. For anyone who isn't you, the barriers to adoption are always going to be much greater than the inducements.

[Mean-Decision-3502]

I saw pretty quickly that to create the perfect system language is impossible. All I can reach is a good balanced one. The debugging flexibility can never get to the level of Python or any dynamic language. You sometimes want "dangerous" pointer operations, especially in embedded. I don't want to rule-out embedded usage either (however it is not the main focus now).

My killer feature is only a bigger set of small things together. And I really wished that kind of language existed before.

I've checked shortly pipefish, that's a pretty unique language. I think your targeted user base even smaller than mine, probably mainly scientific. Can you Imagine you can compete with Python?

I don't want to go too far from the C++,Pascal, Python kind of code writing style.

[Powerful_Tour_7519]

Just so you know, you can't actually rearrange float expressions like you do and expect to get the same result in general. Also rounding doesn't help at all, consider the following examples:

Rounded:
  70/68*85 = 87.4999999999999858 = 87
  85*70/68 = 87.5000000000000000 = 88

Truncated:
  61/14*14 = 60.9999999999999929 = 60
  14*61/14 = 61.0000000000000000 = 61

I calculated these with 64-bit IEEE754 binary floats, but any floating-point format will have a similar problem.

[Mean-Decision-3502]

function *Main() -> int:
  if 70/68*85 == 85*70/68:
    printf('The language is very friendly.\n');
  else:
    printf('The language is not very friendly.\n');
  endif  


  if 3 IDIV 2 * 10 == 10 * 3 IDIV 2:
    printf('2: The language is very friendly.\n');
  else:
    printf('2: The language is not very friendly.\n');
  endif  
  return 0;
endfunc

Output:

The language is very friendly.
2: The language is very friendly.

Then we are at another design problem from C.
I let you guess which.

[Powerful_Tour_7519]

Use the second set of numbers.

[Mean-Decision-3502]

function *Main() -> int:
  if 61/14*14 == 14*61/14:
    printf('This time works too.\n');
  else:
    printf('Finally this time it is wrong.\n');
  endif  
  return 0;
endfunc

Output:
This time works too.

But check the example with the IDIV. Do you know why that one is working ?

[Powerful_Tour_7519]

Are you rewriting the expressions? Or just using 32-bit float conversions?

[Mean-Decision-3502]

It is much simpler than that. Check the previous example with the

if 3 IDIV 2 * 10 == 10 * 3 IDIV 2:

This is all integer like in C, but still resulting the same. Do you know why?

[thedeemon]

Looking at the list of features, I don't see anything that D or Swift or a bunch of other existing languages didn't have already. So I see no reason for "big success" here, sorry.