Cecil

When I started writing vbnc, there weren't many options when it came to deciding which library to use to write the assemblies, only System.Reflection.Emit (SRE) was a real option.

SRE is very powerful, but unfortunately it has a few known and unknown limitations, mostly because it was never designed to be used by a full-fledged compiler.

Since I heard about Cecil, I've wanted to switch, and for the last months I've slowly added support for emitting assemblies with Cecil. Yesterday I reached a very important milestone: vbnc is able to bootstrap itself when using Cecil!

And I have to say that Cecil is A LOT easier to use than SRE, especially with generics.

There are still some problems for switching right away to only using Cecil, one of the biggest being that it's quite slow if you have many referenced assemblies (since Cecil loads everything from an assembly when loading it), though there is work in progress here.

Hopefully these issues will be solved shortly, and I can finally remove everything SRE-related, which has caused me quite a few head-aches!

VB 10

Paul Vick asks what people want for VB 10.

Here's my list:

1) Inline and multiline comments (why not just copy C's /* */?)

2) Option Overflow On/Off

3) Option Warning 30XXX On/Off/Error

3) The ability to apply Options to methods and/or blocks of code. Something like this maybe:


Block Option Strict Off, Option Overflow On
    'Some code here
End Block


I don't particularily like this syntax though, it's somewhat verbose.
Another idea might be to add it to the attribute syntax:


<Option Strict Off> _
Sub A
    <Option Overflow On> _
    Using x
    End Using
End Sub>


4) Select on types, like this:


Select TypeOf obj
    Case Integer
    Case Double
End Select


6) Select on reference equality


Select var Is
    Case objA
    Case objB
End Select


The proposed syntax here doesn't work:


Select obj   
    Case Is varA   
    Case Is varB
End Select


since this is already correct VB code. You can actually write this:


Select number   
    Case Is 1   
    Case Is 2
End Select


where the 'Is' works like a '='. The IDE will remove the 'Is', but it's still allowed. This is bad design IMHO, VB should never have allowed this since it's mixing the meaning of 'Is' and '='.

7) Binary numbers


Dim i As Integer = &B0110110


8) Static method / operator constraints on generic type parameters


Class Point (Of T {+, -})
End Class


Yeah, I know, this is a IL restriction, not a VB restriction, but still...

9) Option Warning 4XXXX On/Off/Error

10) The possibility to specify that sections of code should not be pretty formatted.


#Format None
Dim var As String ()() = new String () { _
            {"a",           "bbbbbb"}, _
            {"cccccccc", "ddd"}}
#End Format


Once in a while it is nice to be able to format the code just as you want yourself, especially when you can detect bugs due to things not being symmetric or lined up.

11) Do NOT allow the user to change pretty formatting (other than disabling it). It's worth it if it avoids the mess in the C world of what's correct formatting or not (how many spaces are there in a tab? 4, 8, 2, 128? where do we put the semi-colon? where do we put spaces?). And mixed formatting in source code is just plain ugly.

Killer feature

I see the light at the end of the tunnel.

Finally, the one single thing that has been annoying me for a loooong time with Opera, is about to be fixed. 

I just hope it's really, really killed now.

Oh Captain My Forms

Today I finally committed support for one of the last remaining features in vbnc: the My namespace.

It was easier than expected, hadn't it been for one little problem I had with a little close friend of mine: Reflection.Emit. I stumbled upon the single worst issue I had during last year's SoC, an issue it took me four days to work around. Note: work around, not fix.

At that time I was able to work around it, since it was happening with the compiler, now this wasn't an option anymore, since it was the generated My code that was causing this.

The problem is quite simple: anytime you have a generic type parameter on a method with a constraint, some other generic type parameter (on a completely unrelated type, not method) might fail emission (with a nice TypeLoadException, claiming that the constraint isn't met).

The workaround I used last year is quite simple: remove all constraints, but as mentioned, that isn't an option anymore, so I started investigating. Unfortunately I lost the test case I cooked up last year, but it didn't take me long to recreate one this time. (Last year I had to start with the entire compiler and comment it out piece by piece...)

Namespace OhCaptainIMixedUpTheTypeParameters
    Class C1(Of Y)
    End Class
    Class C2(Of Z)
        Inherits C1(Of Z)
    End Class
    Class C3
        Sub M1(Of A As New)()
        End Sub
        Sub M2(Of B)()
        End Sub
    End Class
End Namespace

Looks quite innocuous, doesn't it?

On MS this fails with: TypeLoadException: GenericArguments[0], 'B', on 'TypeParameter1.C1`1[A]' violates the constraint of type parameter 'A'.

Now read the exception again and try to relate the A and B to the Z and Y.

Of things you can try to make this compile includes changing the order (for example make C3 the first class in the file), or moving the methods to either C1 or C2.

With this test case in hand I was quite certain it was a bug in Reflection.Emit, but I wasn't entirely sure either. So I added code to vbnc that dumps out how it emits everything in VB code, and got a nice 90-line source file that when compiled and run nicely reproduces the problem.

The conclusion is that it's not a bug in vbnc, which is quite bad (I can't fix it). So everybody that wants to compile any VB code with generic type method parameters, be warned: if the compiler quits with a weird TypeLoadException, you know who to blame.

The best part is that it Mono does not suffer from this bug, so if you want your project to compile no matter what, you'll have to use Mono :)

PD: Already filed a bug with MS (link), and they seem to have confirmed it's a bug in Reflection.Emit.

Update 01/07/2007: Fixed bug-link.
Update 21/09/2011: Fixed html for VB code.

Ain't Numeric

Today I learned something: don't trust a compiler. I repeat: don't ever trust a compiler. It might be doing some magic behind your back.

A while ago I was assigned a bug, at first it looked really easy, but no matter what I tried I couldn't reproduce it, and I was positive the reporter was experiencing the problem. So I just left the bug standing there, and of course, eventually it hit me.

Quiz: If you compile the following source code, what will the IL look like?

Class AintNumeric
    Shared Sub Main
        Console.WriteLine(Microsoft.VisualBasic.Information.IsNumeric("0"))
    End Sub
End Class

The problem was that this printed "False" on Mono and "True" on MS. I compiled the code with vbnc, ran the code on Mono and it printed "True". I ran it on MS and it printed "True". I disassembled the program to see if the compiler was doing something wrong, and it wasn't. I put Console.WriteLines in MS.VB.Information.IsNumeric and they all confirmed that the function was working like it should. Just to check I compiled the program with vbc, ran it on MS and it printed "True".

You might have seen what I missed, but I'll tell you anyway: I didn't run the MS compiled program on Mono. My bad, the reported of the bug explicitly wrote that he did it, but I honestly didn't think it would change anything. vbnc was compiling everything correctly, wasn't it? 

Well no, and now here is the answer to the quiz:

.method public static void Main() cil managed {
    .entrypoint
    .maxstack 8
    L_0000: ldstr "0"
    L_0005: call bool     [Microsoft.VisualBasic]Microsoft.VisualBasic.CompilerServices.Versioned::IsNumeric(object)
    L_000a: call void [mscorlib]System.Console::WriteLine(bool)
    L_000f: ret
}

The Microsoft VB8 compiler changes every method call to Microsoft.VisualBasic.Information.IsNumeric to Microsoft.VisualBasic.CompilerServices.Versioned.IsNumeric. And of course, that function was buggy in Mono...

Now you may think about it and find this is some really bad behaviour, but it's all because of backwards compatibility. When VB 8 came out they added support for unsigned types, which is the reason behind this: The old version of IsNumeric returns false for unsigned numeric types... and since that couldn't change it because it would break backwards compatibility, and they didn't want to add another function handling the new data types (and telling everybody they had to use the new funcion), they created some compiler magic.

And yes, the bug is fixed and vbnc does the same magic now.

How?

A few people have been asking me how to learn how to write compilers.

After all, when I studied they didn't require me to write a compiler, but they didn't teach much about how to write them either, so how did I do it?

I went to the university library and got the Dragon Book. But I didn't have time to read it before I had to return it, so I can't say I've actually read it though. Since I was still interested in writing a compiler when I had to return the book I figured I had to get some documentation that could stay with me (and at the time I didn't have the money to buy any book) so I yahooed a bit and found some resources about recursive descent compilers and printed it out (the original site I used was of course not wikipedia, but it doesn't seem to be available anymore, at least I can't find it - it was some university course in a South African university if I remember correctly).

So, armed with some a little bit of knowledge and much patience I started writing, and here I am today, with a compiler that's actually working on such weird architectures as ppc or s390x. 

It sounds easy and for me it wasn't really that hard (the head-aches I got a few times were usually because of the buggy and lacking Reflection.Emit from MS - this could probably have been avoided had Jb Evain written Cecil a few years earlier).

However I certainly didn't think I would get this far when I started!

Why?

Why did I write a compiler?
When I was studying computer programming at Universidad de Belgrano, one of my fellow students mentioned that "at least they don't require us to write a compiler anymore", making reference to earlier years when the students seemingly were required to write compilers in order to finish their degree.
This obviously made me want to write a compiler, not to finish my degree, but to only satisfy my geeky desire to do something nobody else wanted to do.

Why did I write a compiler for Visual Basic.Net?
Visual Basic was the first computer language I learnt (way back at high school in '98 I got my hands on VB4, later on it was upgraded to VB5 and then VB6), and I did actually like it (no comments on this one please :) .
Later on I started learning C++ (but I didn't like it since the relation between "tangible results"/"amount of code written" is way lower than for VB - in other words it took me far longer to do the same thing in C++ than in VB), and Delphi (which after about half an hour of trying things out the compiler started throwing internal errors at me, so I figured Delphi wasn't the language for me either).
This happened about the same time as the first beta of VS2002 came out (VB7/VB.Net), I had ordered a copy so I was trying it out and learning about the new VB.Net language, and I liked it way better than the previous editions of VB, so the choice wasn't really that hard. Besides, I couldn't find any other VB compiler (non-MS and compatible with MS' version of VB) out there, so it always feels good to create something new :)
The original intention was to write a compiler that compiled VB.Net into native code (I did actually get it produce a HelloWorld.exe, it was however mostly hardcoded into the compiler), but when I got to learn more about the managed world I realized that the intelligent thing to do would be to create a compiler that compiled IL to native code and therefore targeting several languages in one hit. The amount of document-reading required to do this (I would have to learn IL, assembler, the PE file format...), made it completely uninteresting, so I rescoped my project into creating a compiler to compile VB.Net into IL, just as vbc.exe does it.

Why did I write the compiler in Visual Basic.Net?
It didn't start out like that, I first tried C++ (yes, even though I didn't like it, but I thought it was a great change to learn it better and maybe make the mentioned result/work relation better). You can actually still see this in the code (check out the comments at the end of this file). The effort didn't last much though, after a couple of days of debugging memory leaks and seeing weird compiler messages all the time I got bored and figured I'd have to change the source language if I was ever going to finish anything at all.
Next try was VB6. I'd read everywhere that no sane person would write a compiler in VB6, so I decided to kill that myth. After a couple of hours the myth had survived. The lack of inheritance in VB6 made me crazy, I saw myself either copy-paste huge amounts of code, or write it all in functions, with no OOP whatsoever, neither very pleasant alternatives. Especially now that I was learning the new Visual Basic.Net language. Once again, the choice wasn't really that hard.