Tag Archives: Flex

Adventures of a Programmer: Parser Writing Peril XXXX

The final[1] grammar together with the lexer. The whole file with a lot of comments is on Github. It is currently in one file for convenience but needs to be split into a lexer (Flex) and a parser (Bison) later.

The file has a lot of comments already, not much to say here, but nevertheless… Continue reading

Adventures of a Programmer: Parser Writing Peril I

As menacedpromised in the last post: a description of the long path
to a well determined end.

Goal: Writing a small parser for a simple but complete language as a front-end for a for a calculator back-end. With emphasis on “complete”. A simple task one might think, a very simple one. One that could have been described in full in some book I mentioned in another post.

So in medias res, as Caesar wouldn’t have said (he spoke Greek) and of to the very beginning: The Plan. Not that I ever saw it but it is said that The Plan cometh at every beginning.
What is needed for the already briefly described…uhm…purpose, to avoid the repetition of the infamous two words, that are unbeknownst to many programmers, what does the parser need to parse? What do we need to do to enable the poor thing to do so? Continue reading

On Technical Books and Tutorials, Another Unfair Rant

It can get quite frustrating when you want to do something you do know nothing about but are willing to learn it. Not necessarily able but willing. In the times of fast and cheap internet connections the first idea is to search for a small tutorial to get the mere basics. Just enough to know if the tool fits the task and more so: if the tool fits your hand.
Now, what I want to do is to write a different frontend to some well established libraries to juggle numbers. The existing frontends are good, that is not the problem, but they just don’t fit my hands very well. A matter of personal taste, so to say. It is just a frontend, so nothing more than a parser needs to be written and some code to connect the new front end to the backend.
Some faint memories pointed to Flex/Bison and some short glimpses at a couple of basic online tutorials assured me: it is a better choice than writing my own.
The O’Reilly book about it set me short of 30 bucks.
As good as it is but that is too much money for it. It wants too much with an insufficient number of pages. I bought a new copy of Don Knuths’s TAoCP second volume for 29.99 just two years ago because the old just, let me put it blunt, faded away. Original price is AFAIK about 70 bucks but it definitely has more than twice the amount of information in it than the parser-book.

What nags me most in the online-tutorials and also, to some extent, the tech. books is the attitude of handwaving. When it starts to get interesting and useful it is every
f-ing time something in the line of “the rest follows”, carefully camouflaged as “Exercises”. No, folks, the rest does not follow at all! And it is good for students to do let them find it out on their own and if they get stuck they can ask their teachers because that’s what they get payed for.
But a tutorial and even less so a tech. book is a textbook! No pedagogical fine-tuning is needed, just present the stuff, complete and correct, such that you can go on with the whole bunch of other things waiting in your inbox.

It is not more than a bit enervating if some well-meaning people write a webpage, high in the Google ranks but quite useless beyond the most basic needs. It’s ok.
These are mostly people who just learned that stuff and are proud that they
finally grasped it (I know that feeling very well đŸ˜‰ and want to let the world participate. It’s ok!

What is not ok are books that do the same. When I pay money for a book about
a tool I want to know everything about the tool after I read the book and
not—let me repeat NOT “the rest follows”! Or even better: those dreadful exercises!
Well, to be fair for a change: exercises are not necessarily of the same quality in every book or tutorial. Some are quite good, like the ones in the aforementioned TAoCP, some are not so good and some are plain boring. The worst exercises are those that do not offer the solutions somewhere. No chance to control your work. The single exception should be those where no solutions are known (there are some in TAoCP) and you might have the chance to get rich and famous if you solve them.

So, exercises are not bad per se but more apt for a textbook for students who have the time to learn the stuff the old fashioned way. It is not ideal in a book that is the equivalent of a user manual, especially when no solutions to the exercises are given somewhere. Anywhere.

Another nagging detail is the incompleteness of the examples, they start simple, which is the way it should be, of course, but they stay so and ignore a large amount of the capabilities of the tool they try to describe. Or the examples get very special and are of very limited general use. Which is not bad in and of itself but with a very restricted amount of available pages the paper and ink used for it is wasted.

A book about a lexer and parser generator should describe the way to write a parser for a general language. One that has numbers and strings and functions and variables and loops and jumps and scopes and all that and is a good basis for your own project, a foundation you can build skyscrapers upon. You paid for the book and it was not just some small change you paid for it, so you might think you get the complete picture if a complete picture was advertised.

[Formulating the text for this paragraph is left as an exercise for the reader]

This is the end of this rant and this rant only and the start of a loose series describing my adventures in parser-building land.

Next: constructing a grammar of a language that fits my needs. Or something completely different, who knows. I don’t.