Git & make

One of the most annoying issues with git (which is actually a side-product of the fact that it's very easy to create/administer branches), is that switching branches will cause a lot of recompiles.

ccache helps some, but the real life-saver here is a script which doesn't come bundled with your installed git, but which you'll have to download the source code to get: git-new-workdir. It clones your repository to a different directory, with some symlink magic so that everything is shared except the index. It's the git equivalent a checkout of a svn repository to different directories, except that anything you do in any of the directories is available in all directories.

Git & Whitespace

Recently I've been slowly transitioning from svn to git, though one of the mildly annoying differences is that git likes to colorize trailing whitespace in diffs with a very distracting color:



First try at fixing it: a tiny script to fix the file before committing.

Oops, that introduced a huge number of changes and would mess up history quite a bit, not the way to go.

So I added this to my ~/.gitconfig:

[core]
 whitespace = -trailing-space

and now I can see the important changes in the diffs again.

Cecil Reloaded

For our hack week here at Novell I continued hacking on making vbnc work with Cecil, and yesterday I reached the point where vbnc is able to compile itself using only Cecil (in my previous post I also said vbnc was able to bootstrap, but back then vbnc was still using SRE to do all the hard work, and Cecil just to write the final assembly to disk).

The initial performance problems are mostly gone, it's still not quite as fast as before, but there are also quite a few low-hanging fruit yet in that area. The first successful bootstrap yesterday took 40 seconds (compared to 12 seconds for the normal vbnc), and after a few optimizations I'm now down at 24 seconds. Almost all of that time was gained by allocating less strings, the first non-optimized version allocated ~1.4 GB of strings, and it's now down to 119 MB.

I'm also using a delay-loading mechanism to load data from referenced assemblies on-demand, this is not that visible when you compile a lot of code in one go, but it sure make a difference when compiling thousands of 15-20 line tests one by one.

I'm pretty confident I can get compilation time down to the 12 seconds I had before, especially given that Cecil hasn't been much optimized [1] (not actually used by compilers) before, while Mono's SRE implementation is known to be quite fast.

As a sidenote the actual amount of code in the compiler has decreased, from ~75k lines with the normal vbnc to ~68k for the cecil version.

[1] Sébastien Pouliot corrected me, cecil is used by some know compilers, and probably some unknown too. Some parts of Cecil has also been optimized already.

Cecil

When I started writing vbnc, there weren't many options when it came to deciding which library to use to write the assemblies, only System.Reflection.Emit (SRE) was a real option.

SRE is very powerful, but unfortunately it has a few known and unknown limitations, mostly because it was never designed to be used by a full-fledged compiler.

Since I heard about Cecil, I've wanted to switch, and for the last months I've slowly added support for emitting assemblies with Cecil. Yesterday I reached a very important milestone: vbnc is able to bootstrap itself when using Cecil!

And I have to say that Cecil is A LOT easier to use than SRE, especially with generics.

There are still some problems for switching right away to only using Cecil, one of the biggest being that it's quite slow if you have many referenced assemblies (since Cecil loads everything from an assembly when loading it), though there is work in progress here.

Hopefully these issues will be solved shortly, and I can finally remove everything SRE-related, which has caused me quite a few head-aches!

VB 10

Paul Vick asks what people want for VB 10.

Here's my list:

1) Inline and multiline comments (why not just copy C's /* */?)

2) Option Overflow On/Off

3) Option Warning 30XXX On/Off/Error

3) The ability to apply Options to methods and/or blocks of code. Something like this maybe:


Block Option Strict Off, Option Overflow On
    'Some code here
End Block


I don't particularily like this syntax though, it's somewhat verbose.
Another idea might be to add it to the attribute syntax:


<Option Strict Off> _
Sub A
    <Option Overflow On> _
    Using x
    End Using
End Sub>


4) Select on types, like this:


Select TypeOf obj
    Case Integer
    Case Double
End Select


6) Select on reference equality


Select var Is
    Case objA
    Case objB
End Select


The proposed syntax here doesn't work:


Select obj   
    Case Is varA   
    Case Is varB
End Select


since this is already correct VB code. You can actually write this:


Select number   
    Case Is 1   
    Case Is 2
End Select


where the 'Is' works like a '='. The IDE will remove the 'Is', but it's still allowed. This is bad design IMHO, VB should never have allowed this since it's mixing the meaning of 'Is' and '='.

7) Binary numbers


Dim i As Integer = &B0110110


8) Static method / operator constraints on generic type parameters


Class Point (Of T {+, -})
End Class


Yeah, I know, this is a IL restriction, not a VB restriction, but still...

9) Option Warning 4XXXX On/Off/Error

10) The possibility to specify that sections of code should not be pretty formatted.


#Format None
Dim var As String ()() = new String () { _
            {"a",           "bbbbbb"}, _
            {"cccccccc", "ddd"}}
#End Format


Once in a while it is nice to be able to format the code just as you want yourself, especially when you can detect bugs due to things not being symmetric or lined up.

11) Do NOT allow the user to change pretty formatting (other than disabling it). It's worth it if it avoids the mess in the C world of what's correct formatting or not (how many spaces are there in a tab? 4, 8, 2, 128? where do we put the semi-colon? where do we put spaces?). And mixed formatting in source code is just plain ugly.

Killer feature

I see the light at the end of the tunnel.

Finally, the one single thing that has been annoying me for a loooong time with Opera, is about to be fixed. 

I just hope it's really, really killed now.

Oh Captain My Forms

Today I finally committed support for one of the last remaining features in vbnc: the My namespace.

It was easier than expected, hadn't it been for one little problem I had with a little close friend of mine: Reflection.Emit. I stumbled upon the single worst issue I had during last year's SoC, an issue it took me four days to work around. Note: work around, not fix.

At that time I was able to work around it, since it was happening with the compiler, now this wasn't an option anymore, since it was the generated My code that was causing this.

The problem is quite simple: anytime you have a generic type parameter on a method with a constraint, some other generic type parameter (on a completely unrelated type, not method) might fail emission (with a nice TypeLoadException, claiming that the constraint isn't met).

The workaround I used last year is quite simple: remove all constraints, but as mentioned, that isn't an option anymore, so I started investigating. Unfortunately I lost the test case I cooked up last year, but it didn't take me long to recreate one this time. (Last year I had to start with the entire compiler and comment it out piece by piece...)

Namespace OhCaptainIMixedUpTheTypeParameters
Class C1(Of Y)
End Class
Class C2(Of Z)
Inherits C1(Of Z)
End Class
Class C3
Sub M1(Of A As New)()
End Sub
Sub M2(Of B)()
End Sub
End Class
End Namespace

Looks quite innocuous, doesn't it?

On MS this fails with: TypeLoadException: GenericArguments[0], 'B', on 'TypeParameter1.C1`1[A]' violates the constraint of type parameter 'A'.

Now read the exception again and try to relate the A and B to the Z and Y.

Of things you can try to make this compile includes changing the order (for example make C3 the first class in the file), or moving the methods to either C1 or C2.

With this test case in hand I was quite certain it was a bug in Reflection.Emit, but I wasn't entirely sure either. So I added code to vbnc that dumps out how it emits everything in VB code, and got a nice 90-line source file that when compiled and run nicely reproduces the problem.

The conclusion is that it's not a bug in vbnc, which is quite bad (I can't fix it). So everybody that wants to compile any VB code with generic type method parameters, be warned: if the compiler quits with a weird TypeLoadException, you know who to blame.

The best part is that it Mono does not suffer from this bug, so if you want your project to compile no matter what, you'll have to use Mono :)

PD: Already filed a bug with MS (link), and they seem to have confirmed it's a bug in Reflection.Emit.

Update 01/07/2007: Fixed bug-link.