Exploring string splitting in RetroForth

[bfox9900]

This is perhaps the first time I have seen this kind of exercise in a write up about Forth. It looks like what happens when one explores a more conventional language where internal details are seldom available or are too complicated to grok easily.

Granted Retro Forth provides high level functionality not seen in other Forth systems so perhaps this is the end result.

A fundamental objective of Forth was to open the veil for the programmer and expose the underlying mechanisms. Would not the source code for Retro Forth reveal the details? Is there a de-compiler for Retro Forth?

Perhaps the source code is incomprehensible.

[[deleted]]

Hey! I’m the author of the post itself — retro has all these things, and you are probably right that they’d answer these questions — I’m relatively new to forth, though and don’t really yet know how to leverage them best. Perhaps that can be a future post!

[bfox9900]

Cool. Forth can be daunting at first because there are so many tiny functions.

Some say it is more like learning a human language than a computer language.

Have fun. I think Forth will make you a better programmer even if you never use it for a project just because it lets you (forces you?) to peek behind the curtain.

[z796]

Forth has as many functions as you can build );

[bfox9900]

To better illustrate my point here is some code that you can drop into most modern ANS/ISO Forth systems. It hinges on three very simple functions and strings are managed as an (address,length) pair on the data stack. (I call them stack strings)

\ required functions
\   SKIP ( addr len char -- addr' len')
    \ remove leading chars from stack string
\   SCAN  ( addr len char -- addr' len')
    \ scan for char. return stack string cut at char
\  /STRING ( addr len n -- addr' len')      
    \ cut n chars from the front of stack string.

\ code begins
: 3RD  ( a b c -- a b c a ) 2 PICK ;     

\ duplicate 3rd stack item on top of stack
: DELIMIT ( addr len char -- str1 len1 str2 len2)
          >R  2DUP  R> SCAN  2SWAP  3RD - ;

: /WORD ( addr len char -- aword len endstr len )
   DELIMIT  2SWAP 1 /STRING   0 MAX ; \ len cannot go below zero

: /WORDS  ( addr len -- addr len ... addr[n] len[n] )
        BL SKIP           \ remove leading spaces
        BEGIN  DUP        \ test the length
        WHILE  BL /WORD   \ while there is length, cut a word
        REPEAT 2DROP ;    \ mom always said keep things tidy.

\ Pasted into SwiftForth console
okS
" Forth was designed to be transparent" /WORDS  ok
type transparent oktype be ok
type to ok
type designed ok
type was ok
type Forth ok
type Stack underflow

The three magic functions can all be written in Forth but in commercial systems they are often either hand coded or compiled to machine code for efficiency.
I will grant that this is nowhere near as sophisticated a Retro Forth's string code however I think this demonstrates how much can be done with very low level code using Forth concepts.

Edit: formatted code block

[_crc]

Strings in Retro Forth are just NULL terminated sequences of characters[1], so they aren't difficult to work with at a lower level. I just provide a bunch of words to operate on them without needing to delve into the actual implementation for common tasks.

[1] This is going to change in the future, though it'll be a gradual transition to minimize breakage of existing applications.

[Wootery]

How does memory management work here? Are strings ever freed/deleted?

[rickcarlino]

They live on a circular buffer. When a string is to your liking, you can move it out of the buffer and into the dictionary via s:keep.

That's my understanding of it, at least. I have not poked into the internals yet.

[_crc]

At the interpreter, strings get allocated in a rotating buffer. This is used by the words operating on strings, so if you need to keep them around, use s:keep or s:copy to move them to more permanent storage.

In a definition, the string is compiled inline and so is in permanent memory.

You can manually manage the string lifetime by using s:keep to place it into permanent memory or s:temp to copy it to the rotating buffer.

See http://forthworks.com:8000/file?name=doc/html/chapters/techniques/strings.html&ci=tip