Programming language development

[tech291083]

Hi,

There are many programming languages out there today and I wonder as what is required to put together a fully working programming language? If compared to a human body it has heart, brain and a range of organs to make it functional. What does a programming language need in this regard? All is know is a compiler then IDE etc. Can any one please shed some light on the area?
Thanks.

[wje_lf]

You should find this useful.

[likwid]

A computer is more accurately described as a human body. A programming language is just like a real language. It has syntax (you could also say idioms) and an interpreter or compiler, which is analogous to whatever facilities of your brain interpret or compile language.

[Cabhan]

Well, as far as using a language, you only need some method of producing text (so as to write the code) and a compiler. The compiler takes the code and rewrites it in a simpler language. So for instance, an Assembly compiler takes the Assembly code and rewrites it in a binary that the computer understands. A C compiler takes C code and rewrites it in Assembly (which is then turned into a binary). Perl is interpreted into C and then run.

As far as actually designing a language, this is far more complicated, and there are many rules. At its simplest, a language is what we call a grammar, which is just a description of a language (though certain rules do need to be followed). I'm actually taking a programming languages course now, so let me give you an example. This is a description of a language of mathematical operations (arithmetic expressions):

Program = AExp.
AExp : Operation | Number.
Operation = Operator "(" AExp "," AExp ")".
Operator : Add | Sub | Mul | Div.
Add = "+".
Sub = "-".
Mul = "*".
Div = "/".

What sorts of expressions can we build with this? Well, Program is our starting point. It consists of any arithmetic expression. An arithmetic expression is a recursive datatype that can either be an operation (which contains two arithmetic expressions) or a number. So our expressions can be:

5
+(5, 5)
-(6, 3)
*(5, +(2, 3))

and so on.

The way that most languages are implemented these days is by creating a lexical specification and a grammar, and then using tools to generate a scanner and a parser. The scanner takes the code as input, and breaks it up into tokens. For example, in the above language, a scanner would find "+", "-", "*", "/", "(", ")", ",", numbers, and whitespace. All of these except for whitespace would be passed into our parser (we would tell our scanner to just ignore whitespace, since it doesn't matter here).

The parser would take these tokens, and would follow our grammar to create some sense of it. This is often implemented in an existing programming language. In my class, for instance, we are using Scheme and Java to implement our parsers and interpreters, even though the expressions that we are parsing are not Scheme or Java.

That's basically a very simple explanation of how a programming language would be designed. Once our parser had created some representation of the expression in our implementing language, we could do whatever we wanted. For instance, we might evaluate the expression and return a single number, or we might write a compiler, or any such thing.

It's an interesting field, and one that requires a good deal of thought. Many languages have a formal standard, which describes the language in excruciating detail. For instance, the C99 standard can be read at:
http://www.open-std.org/JTC1/SC22/WG...docs/n1256.pdf

[tech291083]

The way that most languages are implemented these days is by creating a lexical specification and a grammar, and then using tools to generate a scanner and a parser. The scanner takes the code as input, and breaks it up into tokens. For example, in the above language, a scanner would find "+", "-", "*", "/", "(", ")", ",", numbers, and whitespace. All of these except for whitespace would be passed into our parser (we would tell our scanner to just ignore whitespace, since it doesn't matter here).

The parser would take these tokens, and would follow our grammar to create some sense of it. This is often implemented in an existing programming language. In my class, for instance, we are using Scheme and Java to implement our parsers and interpreters, even though the expressions that we are parsing are not Scheme or Java.

That's basically a very simple explanation of how a programming language would be designed. Once our parser had created some representation of the expression in our implementing language, we could do whatever we wanted. For instance, we might evaluate the expression and return a single number, or we might write a compiler, or any such thing.

It's an interesting field, and one that requires a good deal of thought. Many languages have a formal standard, which describes the language in excruciating detail. For instance, the C99 standard can be read at:
http://www.open-std.org/JTC1/SC22/WG...docs/n1256.pdf

Well done sir,

What a fine explanation of the whole concept. Excellent. I am very happy with the reply you have just provided to me. All I can say is thank you.

[tech291083]

Thanks you gus all have really replied very interesting and thoughtfully. Great.