JustDecompile – First Impressions

I have posted a new review of JustDecompile that looks at a later beta release.

With Red Gate now charging for .NET Reflector and updating the free version to expire at the end of May I’ve been starting to think about alternate decompilation tools.  A few days ago I saw an article on InfoQ about Telerik’s JustDecompile tool and thought I’d take a look.  So how does JustDecompile fare?

At the time of this writing JustDecompile is still in beta.  The version I had was Beta 2011.1.516.2.  Anything I comment on here is definitely subject to change and what I’m writing about are my first impressions of the product.  Unless otherwise noted I’ve reported every issue I’ve encountered to Telerik.  Getting this involved in their beta program is kind of my way of saying thanks to them for sponsoring so many of the events I’ve attended and found value in over the years.

JustDecompile has a pretty clean UI.  Most of the functionality is exposed through a simple toolbar rather than buried under a menu structure which is a nice touch.  Some of the (I guess) lesser used options are found in a hidden toolbar that can be displayed by clicking a down arrow that’s about 1/2 as tall as a standard menu so I’m not really sure why a menu wasn’t used instead.

Opening an assembly is just a matter of clicking the “Open…” button or hitting Ctrl+O to browse for the file.  Once the assembly is open it is presented in a tree control on the left side of the window and works about as I expected it to.  JustDecompile also presents some decent metadata such as the fully qualified name and target framework.  I’d like to see a bit more information such as whether the assembly targets x86 but overall I have very few complaints here.

One shortcoming I’d really like to see addressed (there’s an open suggestion for it in the forum) is to either add a manual reload option or a FileSystemWatcher to automatically reload assemblies as they change.  Constantly closing and reopening the assembly as I was experimenting with the decompiler got tiresome in a hurry.  I believe that a manual reload option is the best route but a configuration setting would be a nice half-way point.

The Go To Type and Go To Symbol searches are OK at best.  At first I thought it would be nice to consolidate them into a single search dialog but then I wondered why the distinction is even made.  Why can’t we just search and possibly filter if the distinction is really that important?  It would also be nice if the results were presented in a docked panel rather than in the dialog itself so we could click from one result to the next without having to reopen the dialog but that’s a minor issue compared to some other things I encountered.

First, both dialogs duplicate results.  JustDecompile waits for you to stop typing for ~1 second before it searches.  I found that if I typed a few characters, waited, typed a few more characters and waited again that the new matches were just added to the list without clearing it first.

The bigger problem I had here was that I could crash the application by trying to click between the two dialogs.  I clicked the Go To Type button to open the dialog, clicked the Go To Symbol button, noticed the dialog didn’t change, then clicked the Go To Type button again to be presented with a HUGE stack trace followed by a “JustDecompile has stopped working” dialog box.  This behavior was repeatable each and every time.

So far we’ve talked exclusively about the UI and most of the issues have been pretty minor (except the crash, of course) but how well does it do what it’s supposed to do?  Before going any further let’s address a few assumptions:

  1. All assemblies tested were originally written in C#.
  2. All decompilation tested was to C#.  Even though JustDecompile supports VB.NET I’m not proficient enough in it to gauge the validity of the output.
  3. The code I tested was nothing out of the ordinary nor was I trying to “trick” the tool.  I can easily see myself writing or encountering code similar to the samples on a day-to-day basis.

I had high hopes for this product but I quickly came to the realization that JustDecompile simply isn’t usable.  More often than not I found that the code generated by JustDecompile was, to put it mildly, unreliable.  So many of my tests resulted in invalid C# or deviated from the original source so much that I determined the output couldn’t be trusted.

For this assessment I created a simple class with a single method and an enum with three items.  I tweaked things here and there to try different variations of the same basic concepts.  The two areas I focused on before giving up on the tool were switch statements and the conditional operator (?:).

Each of the following samples assumes the following enumeration definition:

public enum SampleEnum
{
	Unknown,
	Item1,
	Item2
}

A quick note about enumerations – when I first started looking at JustDecompile I loaded a few existing assemblies and saw some inconsistent behavior with how JustDecompile handled them.  Sometimes the decompiler showed the actual enum item, other times it showed the underlying integer value.  I wasn’t able to figure out a pattern nor could I determine a way to reproduce it so I haven’t formally reported it.  In these examples the enum item was always shown.

I started with switch statements.  My first test was a switch statement that simply returned a string based on an enum value.

public string SampleMethod(SampleEnum enumValue)
{
    switch (enumValue)
    {
        case SampleEnum.Item1:
            return "Item 1";

        case SampleEnum.Item2:
            return "Item 2";

        case SampleEnum.Unknown:
            return "Unknown";
    }

    return String.Empty;
}

JustDecompile handled this simple block pretty well. The only thing I really noticed was that it switched the order of the conditions and included curly braces but at least it’s correct. This was the first and last of my tests where this was true.

public string SampleMethod(SampleEnum enumValue)
{
    switch (enumValue)
    {
        case SampleEnum.Unknown:
        {
            return "Unknown";
        }
        case SampleEnum.Item1:
        {
            return "Item 1";
        }
        case SampleEnum.Item2:
        {
            return "Item 2";
        }
    }
    return string.Empty;
}

My second test was a variation of the first where each condition set a variable rather than returning a string.

public string SampleMethod(SampleEnum enumValue)
{
    var val = String.Empty;

    switch (enumValue)
    {
        case SampleEnum.Item1:
            val = "Item 1";
            break;

        case SampleEnum.Item2:
            val = "Item 2";
            break;

        case SampleEnum.Unknown:
            val = "Unknown";
            break;
    }

    return val;
}

I expected to see some difference between the original source and the decompiled version but what I saw astounded me.

public string SampleMethod(SampleEnum enumValue)
{
    string empty = string.Empty;
    switch (enumValue)
    {
        case SampleEnum.Unknown:
        {
            empty = "Unknown";
        }
        case SampleEnum.Item1:
        {
            empty = "Item 1";
        }
        case SampleEnum.Item2:
        {
            empty = "Item 2";
        }
    }
    return empty;
}

There are two things wrong with this code. Have you found them yet? The biggest problem is that the code won’t compile. If we tried to compile this result we’d be scolded with the message “error CS0163: Control cannot fall through from one case label (‘case 0:’) to another,” The other problem isn’t quite so significant but I feel that it’s worth mentioning.

JustDecompile ignores the names of the local variables despite their presence in the .locals init section of the MSIL. Although this isn’t necessarily a problem per se it certainly doesn’t lend itself to producing readable code. “empty” certainly isn’t always empty in the generated code.

After seeing how JustDecompile choked on a simple switch statement I thought about stopping there but now I was curious to see how it would handle a few other common scenarios. For the next test I simply replaced SampleEnum.Unknown case with default and things started getting really scary.

public string SampleMethod(SampleEnum enumValue)
{
    var val = String.Empty;

    switch (enumValue)
    {
        case SampleEnum.Item1:
            val = "Item 1";
            break;

        case SampleEnum.Item2:
            val = "Item 2";
            break;

        default:
            val = "Unknown";
            break;
    }

    return val;
}

There’s nothing out of the ordinary here but what does JustDecompile produce?

public string SampleMethod(SampleEnum enumValue)
{
    string empty = string.Empty;
    switch (enumValue)
    {
        case SampleEnum.Unknown:
        {
            empty = "Item 1";
        }
        case SampleEnum.Item1:
        {
            empty = "Item 2";
        }
        default     :
        {
            empty = "Unknown";
        }
    }
    return empty;
}

Ignoring the formatting problem for the default block we can see that the generated code is just wrong. Aside from suffering from the same compilation problems as the previous example, the cases aren’t even the same as the test code; it switched the cases around but not the case bodies! JustDecompile produced code that in no way represents the original source!

There’s also another bug around default statements.  I found that if I replaced the body of the default statement with a method call:

public string SampleMethod(SampleEnum enumValue)
{
    var val = String.Empty;

    switch (enumValue)
    {
        case SampleEnum.Item1:
            val = "Item 1";
            break;

        case SampleEnum.Item2:
            val = "Item 2";
            break;

        default:
            Console.WriteLine("Unknown");
            break;
    }

    return val;
}

…JustDecompile also includes the MSIL label and a goto statement!

public string SampleMethod(SampleEnum enumValue)
{
    string empty = string.Empty;
    switch (enumValue)
    {
        case SampleEnum.Unknown:
        {
            empty = "Item 1";
        }
        case SampleEnum.Item1:
        {
            empty = "Item 2";
        }
        default     :
        {
            Console.WriteLine("Unknown");
            goto L_0038;
        }
    }
L_0038:
    return empty;
}

So we’ve seen how JustDecompile mangles pretty basic switch statements but how does it do with the conditional operator? Unfortunately, it mangles those to the point where they’re unusable too. These tests simply assign an enum value based on whether an integer is even.

In the first form I used a variable for the conditional:

public void SampleMethod(int arg)
{
    var isEven = arg % 2 == 0;
    var val = isEven ? SampleEnum.Item1 : SampleEnum.Item2;
}

Once again, there’s nothing tricky here but JustDecompile seems to think otherwise:

public void SampleMethod(int arg)
{
    bool flag = (arg % 2) == 0;
    SampleEnum sampleEnum = flag || SampleEnum.Item2;
}

This is another case where JustDecompile produces code that won’t compile! Attempting to compile this code results in the message: “error CS0019: Operator ‘||’ cannot be applied to operands of type ‘bool’ and ‘JustDecompileTests.SampleEnum’,” There’s also no mention of SampleEnum.Item1 in the output! Of course there’s also no mention of the conditional operator either… At least it managed to get the conditional variable assignment right even if it gave it a nonsensical name.

Inlining the condition with the conditional operator makes matters even worse:

public void SampleMethod(int arg)
{
    var val = arg % 2 == 0 ? SampleEnum.Item1 : SampleEnum.Item2;
}

results in:

public void SampleMethod(int arg)
{
    SampleEnum sampleEnum = !arg % SampleEnum.Item2 || SampleEnum.Item2;
}

Taking the modulus of a negated int and an enum? Throwing out both the conditional operator and one of its arguments? It was at this point I decided to stop testing. If JustDecompile can’t handle even these simple examples how can I trust it to get “real” assemblies correct?

As I mentioned I had high hopes for this product when I started investigating it but unfortunately it has proven to be unusable by producing results that are totally unacceptable. Hopefully Telerik will address these issues for the release. I’ll be willing to give it another look after it has matured a bit but for now it has no place in my toolkit.

Advertisements

8 comments

  1. Thanks a bunch for spending time on reviewing JustDecompile. Much appreciated, indeed.

    It looks like you had stumbled upon the most problematic points in the JustDecompile. Overall, we’d like to think the tool is not as useless as your tests show. Normally, it’d work just fine in the majority of scenarios. That’s why we released the beta after all. It would not make much sense to reelase a tool that’s mostly useless.

    Oh, also, thanks for taking the time to report these issues to us. Luckily, we were already aware of these problems and we’ve already been working on fixing them. So, the fixes are on their way. I’ll get back to you once we have a new build that passes your tests.

    Once again, thanks, and don’t hesitate to let us know should you have any further problems.

    Tsviatko Yovtchev
    Telerik

    1. Thank you for taking the time to read the review and comment. It’s nice to know that the feedback has reached the people that care the most about it.

      Your response prompted me to take another, closer look with some existing .NET 2.0 assemblies that have been in use for years. I wanted to give the tool the benefit of the doubt but came away even more concerned than before. I found that JustDecompile usually produced valid results for the simplest of methods that consisted mainly of assignment operations and some basic calculations. Unfortunately, what JustDecompile did to one method I looked at that only contains one line (a return of the logical or of three comparisons, no enums involved) couldn’t pass a unit test to find the main value it was trying to detect!

      I looked at another, more complicated method in the same class and I wasn’t sure I was even looking at the same code. Here’s a list of what I observed:

      • it assigned values of one type to variables of another incompatible type
      • it introduced an if statement that was not present in the original source nor could I determine its origin
      • used undefined variables
      • Tried to negate an int
      • alternated between enums and integers in the switch cases – as a side note, the integers it used aren’t even valid for the enum
      • failed to return a value

      I fully understand and appreciate not wanting to think of the tool as useless but I’ve seen time and time again that the output can’t be trusted. If I know that JustDecompile produces invalid results I have to question all of its results and that severely limits its usefulness.

      1. That’s weird. I am not saying we are covering all the corner cases correctly but the amount of trouble you have leaves me asstounded. Could you send over the assembly where you encountered these problems together with a list of method names to look at (I assume you are able to see the e-mail address I used to submit this post)? If that’s not possible, it’d be really nice if you could send over the MSIL listings of the methods in question. That’s definitely something we want to have a look at.

        1. I do see your email address in the comment list so I can send you something this evening. The assembly is mostly some utility classes so I think I can send it but if not I can definitely get you the MSIL.

          Thanks!

    1. Tim:

      Thanks for the note. I’d like to remind my readers that this was written against an early beta of JustDecompile. The folks at Telerik have released a few updates to since I wrote this article but I haven’t had a chance to write an update yet. The last version I checked handled the switch statements correctly and used the variable names as defined in .locals in the MSIL but was still choking on the conditional operator and underlying Enum values. I’m hoping to get a chance to update my findings in the next week or so.

Comments are closed.