ChubakPDP11

For anyone seeking to write their own database, I have one recommendation: the Tokyo Cabinet Library.

Tokyo Cabinet abstracts away all the needs of writing your own serializers and deserilizers for binary formats. You can have hashtable databases , B+ trees and everything else all prepared for you under one roof.

Of course that is if you have brain, and don't use a text storage format like JSON. If you use shit like JSON and YAML

and add potentially hundreds of millisceonds of parsing time just to serialize data from text into machine-readable binary, then please submit your name and address to my council so we can get rid of you when we own the world.

Cleansing of the undesirables aside (seriously, give me ONE good thing about text storage formats! They are EXCHANGE formats, not STORAGE formats!), Tokyo Cabinet is written in C so you can easily bind it with SWIG. But there's probably bindings around if you look.

I remember reading something about recursive languages in Aho's Automata Book. I will have to check it again. So Regular Languages can't be recursive? Is that what 'recursive' language even means? I have to read the book again.

Thanks for your help man.

btw this is the ASDL I wrote:

%{

#include "mukette.h"


static Arena *ast_scratch = NULL;


#define ALLOC(size) arena_alloc(ast_scratch, size)


%}

md_linkage = Hyperlink(md_compound? name, string url)
	  | Image(string? alt, string path)
	  ;


md_inline = Italic(md_compound italic)
	 | Bold(md_compound bold)
	 | BoldItalic(md_compound bold_italic)
	 | Strikethrough(md_compound strike_through)
	 | InlineCode(string inline_code)
	 | Linkage(md_linkage linkage)
	 | RegularText(string regular_text)
	 ;


md_header_level = H1 | H2 | H3 | H4 | H5 | H6 ;


md_line = Header(md_compound text, md_header_level level)
	| Indented(md_compound text, usize num_indents)
	| LinkReference(identifier name, string url, string title)
	| HorizontalLine
	;


md_compound = (md_inline* compound) ;


md_table_row = (md_compound cells, size num_cell) ;


md_table = (md_table_row* rows, size num_rows) ;


md_ol_item = (int bullet_num, md_list_item item) ;


md_ul_item = (char bullet_char, md_list_item item) ;


md_list_item = TextItem(string text)
	    | OrderedItem(md_ol_item ordered_item)
	    | UnorderedItem(md_ul_item unordered_item)
	    | NestedList(md_list nested_list)
	    ;



md_list = (md_list_item* items) ;



md_block = Pargraph(md_compound* paragraph)
	| BlockQuote(md_compound* block_quote)2
	| CodeListing(identifier? label, string code)
	| Table(md_table table)
	| List(md_list list)
	| Line(md_line line)
	;


markdown = (md_block* blocks) ;


%%


static inline void init_tree_scratch(void) { ast_scratch = arena_init(AST_ARENA_SIZE); }
static inline void free_tree_scratch(void) { arena_free(ast_scratch); }

I had an easier time parsing ASDL with Yacc. I still can't tell whether a grammar is LR, LL or RE, but I can tell that Markdown is not CFG.

I just updated ASDL: https://github.com/Chubek/ZephyrASDL

Apologies if I am too late on the documetnation. I am still trying to improve it by using it myself. I also wish to add an Ocaml target.

So I'll answer your question and ask a question back from anyone who can help me.

RE the nesting, I was under the impression that they can't be combined when I made it. Then I read CommonMark's specs and it seems like it's possible. It would be miserable to do this with a syntax-directed translation. I used ASDL to write up a tree, and added some features to asdl(1) so they would be handled more properly. I am not sure whether I should use a parser generator for this, but the nesting can be handled by Lex's start conditions

if I fail to do so, I may use a PEG generator.

Now my question.

I think nesting and recursion are a good case for using a push-down parser here

I will still try and find a solution before I use an LR parser.

I avoid using Yacc because I honestly have no clue how to use it with a language like Markdown.

So my thinking is, I would just use a starting condition stack with Lex (I use Flex). It's pretty simple. Let's use a linked list so there are no limits.

struct stack { int state; stuct stack *next,  }

struct stack *top, *bottom;

void push ...

int pop ...

(I usually use typedefs though).

So now we have a psuedo-pushdown parser. What are these called?

I am still a beginner at this but one thing that worries me is, how would I handle the tree with this method?

With Yacc or PEG parser generators, it is easy to assign a value to a reduction/closure. But with this method, Flex won't allow me. Unless I use too many variables.

I think I may use peg(1). I can even do the same stack thingy with PEG.

Any help is welcome.

Why did you use a ? as a prompt for E, but a >>> as a prompt for Python? I know CPython uses >>> in its termio prompt (and I don't know how they brought that to Windows?) but why would have E used ??

Definitely do. Or just pirate it, it does not matter. I pretty much doubt a cent of what you spend on that print or e-print will be given to any of the authors. Mostly likely, they have been paid beforehand, or were just so passionate on the subject, they did it pro-bono. The publisher is not one of those bougie 'boutique' publishers like No Starch Press, it's freaking Cambridge Press, they can take the loss.

I do often buy books from the aforementioned 'bougie' and 'boutique' pub houses (mind the ASCII 32) and that's hard on me because I am cut off from international banking system and have to do it through a courier. But it's still worth it. Like 'Crafting an Interpreter' is a book I would spend money on. Or any of 'Pragmatic Bookshelf's books, they are great. But don't waste your money on this.

I realize reading PDFs could be problematic on e-readers, so I will post a guide about making it easier. Check out my post history some time from now.

Thanks.

I'm a simp for Alan Kay and I goon to his SnappChat, but I subscribe to Andrew Appel's OnlyFans (along with many others), so I did not look much into SmallTalk when it comes to '70s languages'. I guess I should do that. Thanks. I could have never put two and two together to realize Python uses prototypes. This blows my mind. Funny thing is just the other day I wrote JavaScript's grammar. https://gist.github.com/Chubek/0ab33e40b01a029a7195326e89646ec5

I guess I still got a lot to learn so better get moving. I guess by 'full semantic analysis' I meant do a a full type analysis 'before' you emit the bytecode, not after. What is the protocol here exactly? I have seen several variants and supersets of Python that do an ML-style type analysis. They achieve it via the `NAME [ ':' TYCON ]' syntax so the regular Python interpeter would still work.

So thanks. Learned something with your post.

Well I just use the arrow keys like a good IBM-PC cuck boy born in 1993. I mean the literal arrow keys. Keep one thing in mind, a keyboard in the late 70s, probably connected to:

1- A dumb terminal, connected directly to a mini-computer such as VAX or my dear PDP-11, 2- A smart (or dumb) terminal, connected via 120mA current to a central mainframe,

... may indeed HAVE or HAVE NOT arrow keys! Arrow keys are an IBM-PC thing, and even then, some clones omitted them. It was not until the IBM-derived PCs were standardized that we were sure there would be arrow keys on the keyboard.

UNIX was ported over and over to several systems. It was originally on my dear PDP-11, but as people found it the great thing that it was (and still is!) they ported it over to their own systems. This led to its standardization through FIPS-POSIX. Later, Ken Thompson wrote 'pcc' or Portable C Compiler, and porting UNIX became easier. Before that, people used documents like Lion's Commentary on the UNIX Source, which is what Stella's Programmer's Guide was to Atari 2600 is to UNIX.

UNIX was originally designed for mini-computers, and time-sharing mainframes. It wasn't until some Fingoloid whose name I keep forgetting, Linus Tech Tips or something, that a true UNIX for IBM PCs, or rather, 386, or rather, x86 PCs was born.

So why did they just not use arrow keys? Because, I guess!

Well what you are describing are the abstractions upon these models. So the 'basis' still remains true. Think of it this way:

• A modern machine is based on the RAM model;
• The RAM model is based on the register model;
• The register model is based on several lower-level abstractions, at the end of which we get to a Turing machine.

So AFAIK, Sipser's does not explain more intricate models like RAM machine or register machine, I recommend spending some time prompting ChatGPT or reading some books on it (don't know any off hand).

TL;DR: The formal languages described in Sipser's are the lowest levels of abstraction. Higher levels of theoretical abstractions do exist, which I described several here.

At the end, remember that theory and practice are to be used in tandem, neither will achieve anything alone. Modern languages we use today once resembled a formal, theoretical language, a PDA made of PDAs. Look at the earliest C compilers, for example: https://github.com/mortdeus/legacy-cc/tree/master/prestruct. It's just a giant state machine.

I highly recommend picking up an HDL (Hardware Description Language), or implementing one if you got the time. This shall give you a clear picture of how theory differs from practice.

PS: Modern systems languages also never give a true picture of what programming is, and what should it be about. I recommend picking up a language closer to functional than imperative. I recommend OCaml, or Haskell if could be. That is the other side of formality.

Glad to hear that. I am a sucker for S-Expressions a s well. I recently wrote a script in Ruby which syntax-directly translated JSON to S-Expressions. Also, please check out my S-Expression parser in C: https://gist.github.com/Chubek/d2f0ac9067521716d2ab31c93948e885