Creating a Generative Type Provider

In my recently released Pluralsight course, Building F# Type Providers, I show how to build a type provider that uses erased types. To keep things simple I opted to not include any discussion of generative type providers beyond explaining the difference between type erasure and type generation. I thought I might get some negative feedback regarding that decision but I still believe it was the right decision for the course. That said, while the feedback I’ve received has been quite positive, especially for my first course, I have indeed heard from a few people that they would have liked to see generated types included as well.

There are a number of existing type providers that use generated types, most notably in my opinion is the SqlEnumProvider from FSharp.Data.SqlClient. That particular type provider generates CLI-enum or enum-like types which represent key/value pairs stored in the source database.

Although SqlEnumProvider is a great example and is relatively easy to follow, general how-to documentation for generative type providers is hard to come by to say the least. As such I thought that showing how to write the ID3Provider built in the course as a generative type provider would be a nice addendum for the course material. I clearly won’t be covering everything I do in the course here so if you’re looking for a deeper understanding of type providers I strongly encourage you to watch it before reading this article.

Creating a generative type provider is very similar to building an erased type provider but it does require a little extra work. The compiler isn’t quite as forgiving particularly around inline pattern matches in signatures.

Let’s start by looking at the completed generative type provider then working through the pieces relevant to producing generated types.

namespace DidacticCode.ID3

open System.IO
open System.Reflection
open Microsoft.FSharp.Core.CompilerServices
open Microsoft.FSharp.Quotations
open ProviderImplementation.ProvidedTypes

type FrameDictionary = System.Collections.Generic.Dictionary<string, ID3Frame>

[<TypeProvider>]
type GeneratedID3TagProvider(config: TypeProviderConfig) as this =
  inherit TypeProviderForNamespaces()
  let assy = Assembly.LoadFrom(config.RuntimeAssembly)
  let ns = "DidacticCode.TypeProviders"
  let id3ProviderType = ProvidedTypeDefinition(assy, ns, "ID3ProviderGenerated", Some typeof<obj>, IsErased = false)

  let framesField = "_frames" |> makeProvidedPrivateReadonlyField<FrameDictionary>

  let makePropertyBody frame (o: Expr list) =
    let fieldGet = Expr.FieldGet(o.[0], framesField)
    <@@ (%%fieldGet:FrameDictionary).[frame].GetContent() @@>

  let instantiate typeName fileName =
    let ty =
      ProvidedTypeDefinition(assy, ns, typeName, Some typeof<obj>, IsErased = false)
      |>! (Seq.singleton >> Seq.toList >> ProvidedAssembly(Path.ChangeExtension(Path.GetTempFileName(), ".dll")).AddTypes)

    framesField
    |> ty.AddMember

    makeProvidedConstructor
      List.empty
      (fun args -> Expr.FieldSet(args.[0], framesField, <@@ fileName |> ID3Reader.readID3Frames @@>))
    |>! addXmlDocDelayed "Creates a reader for the specified file."
    |> ty.AddMember

    fileName
    |> ID3Reader.readID3Frames
    |> Seq.choose (fun f -> match f.Key.ToUpperInvariant() with
                            | "APIC" as frame ->
                                "AttachedPicture"
                                |> makeProvidedProperty<AttachedPicture> (makePropertyBody frame)
                                |>! addXmlDocDelayed "Gets the picture attached to the file. Corresponds to the APIC frame."
                                |> Some
                            // Additional frames omitted
                            | _ -> None)
    |> Seq.toList
    |> ty.AddMembers

    ty

  do
    id3ProviderType
    |> Seq.singleton
    |> Seq.toList
    |>! (ProvidedAssembly(Path.ChangeExtension(Path.GetTempFileName(), ".dll")).AddTypes)
    |>! (fun types -> this.AddNamespace(ns, types))
    |> ignore

  do
    id3ProviderType.DefineStaticParameters(
      [ ProvidedStaticParameter("fileName", typeof<string>) ],
      (fun typeName args -> instantiate typeName (unbox args.[0])))


[<assembly:TypeProviderAssembly>]
do ()

If you’ve completed the course you should immediately recognize how similar this code is to the erased version but there are a number of important differences.

The first difference is the type provider’s signature.

type GeneratedID3TagProvider(config: TypeProviderConfig) as this =

We still have a self-referencing class but now we’re also accepting a TypeProviderConfig in the constructor. We need this for the next important change:

let assy = Assembly.LoadFrom(config.RuntimeAssembly)

We’ve left the namespace alone but have changed the assembly to the one that created the type provider instance. Per the MSDN documentation, these assemblies will be statically linked into the resulting assembly which ultimately means that the generated types will be defined within the consuming application’s assembly.

The next important piece is setting IsErased to false on the id3ProviderType. We also need to change None to Some typeof to explicitly tell the generator to base the provided type on obj.

let id3ProviderType = ProvidedTypeDefinition(assy, ns, "ID3ProviderGenerated", Some typeof<obj>, IsErased = false)

Note that I changed the provided type name to ID3ProviderGenerated to distinguish it from the erased version created in the course.

With that provided type no longer being erased we need to add it to a provided assembly which we do in the do binding near the end of the code listing.

do
  id3ProviderType
  |> Seq.singleton
  |> Seq.toList
  |>! (ProvidedAssembly(Path.ChangeExtension(Path.GetTempFileName(), ".dll")).AddTypes)
  |>! (fun types -> this.AddNamespace(ns, types))
  |> ignore

For convenience I revised the original do binding to handle adding the type to the ProvidedAssembly and registering it with the namespaces in a single pipeline using the tee operator.

We also need to update the type returned from the instantiate function to be based on obj and not be erased. Because we’re in a function though we can safely just use a side-effecting pipeline with function composition to create the provided assembly and register the type.

let ty =
  ProvidedTypeDefinition(assy, ns, typeName, Some typeof<obj>, IsErased = false)
  |>! (Seq.singleton >> Seq.toList >> ProvidedAssembly(Path.ChangeExtension(Path.GetTempFileName(), ".dll")).AddTypes)

Before we go any further there’s an important change to the instantiate function’s signature. Generative type providers don’t like inline pattern matching and will report an error about incomplete pattern matches so to work around that we change the signature to take typeName and fileName instead.

let instantiate typeName fileName =

This, of course, requires us to change the call site as well:

do
  id3ProviderType.DefineStaticParameters(
    [ ProvidedStaticParameter("fileName", typeof<string>) ],
    (fun typeName args -> instantiate typeName (unbox args.[0])))

Now, instead of simply passing instantiate as the instantiation function we wrap it in a lambda expression and extract the file name from the args array.

Because we’re now generating types rather than erasing them and letting the compiler track the result we need to define a field to hold the values. To keep things consistent with the course I defined another helper function, makeProvidedPrivateReadonlyField to streamline this. makeProvidedPrivateReadonlyField is defined as follows:

let inline makeProvidedPrivateReadonlyField<'T> fieldName =
  ProvidedField(fieldName, typeof<'T>)
  |>! (fun f -> f.SetFieldAttributes(FieldAttributes.Private ||| FieldAttributes.InitOnly))

This is then used to create the field like this:

let framesField = "_frames" |> makeProvidedPrivateReadonlyField<FrameDictionary>

Note that framesField is defined outside of initialize but added to the type produced by initialize. This is because we’ll need the reference in both the makePropertyBody function and the provided constructor’s body.

I could have moved the makePropertyBody function inside of instantiate but I wanted to keep the code consistent with the course so I left it where it was.

First, let’s look at makePropertyBody since it appears first in the original listing.

let makePropertyBody frame (o: Expr list) =
  let fieldGet = Expr.FieldGet(o.[0], framesField)
  <@@ (%%fieldGet:FrameDictionary).[frame].GetContent() @@>

This differs only slightly from the erased version but it’s an important difference. In the erased version we use an inline match in the function signature and splice it into the quoted expression as obj, cast it to FrameDictionary, access the frame via the indexer, and return the content. Here we wrap our provided field (_frames) within a FieldGet expression. The first argument there (o.[0]) identifies the instance so the compiler knows where to get the value. We then splice that expression in and return the content as before.

The provided constructor body is similarly changed. Since we’re now going to set the _frames field to the dictionary returned by our ID3 reader we now just return a FieldSet expression that identifies the provided field and the same quotation as in the erased version.

makeProvidedConstructor
  List.empty
  (fun args -> Expr.FieldSet(args.[0], framesField, <@@ fileName |> ID3Reader.readID3Frames @@>))
|>! addXmlDocDelayed "Creates a reader for the specified file."
|> ty.AddMember

And finally we need to make sure we register the field with the provided type.

framesField
|> ty.AddMember

With these changes in place a consuming application would now use concrete instances of the provided types rather than having the various member implementations injected in place of the members. If you have the consuming application from the course, you shouldn’t need to make changes to the code beyond renaming the type provider to ID3ProviderGenerated. If you were to now build the consuming application and inspect the resulting assembly in ILSpy as we did with the erased provider in the course you should see something like the following (C#):

public static class Program
{
	public sealed class audioSample
	{
		private readonly Dictionary<string, ID3Frame> _frames;
		[TypeProviderXmlDoc("Gets the track title. Corresponds to the TIT2 frame.")]
		public string TrackTitle
		{
			get
			{
				return this._frames["TIT2"].GetContent();
			}
		}
		
		// Additional properties omitted
		
		[TypeProviderXmlDoc("Creates a reader for the specified file.")]
		public audioSample()
		{
			this._frames = ID3Reader.readID3Frames("C:\\Dev\\Presentations\\TypeProviders\\Custom\\SampleFiles\\Sample1.mp3");
		}
	}
	public static void showForm(Program.audioSample sample)
	{
		ImageConverter converter = new ImageConverter();
		byte[] image2 = sample.AttachedPicture.Image;
		ImageConverter imageConverter = converter;
		byte[] value = image2;
		Image image = (Image)imageConverter.ConvertFrom(value);
		try
		{
			Form form = new Form
			{
				BackgroundImage = image,
				BackgroundImageLayout = ImageLayout.Center,
				Height = image.Height,
				Width = image.Width,
				Text = sample.AlbumTitle + ": " + sample.TrackTitle
			};
			try
			{
				DialogResult dialogResult = form.ShowDialog();
			}
			finally
			{
				IDisposable disposable = form as IDisposable;
				if (disposable != null)
				{
					disposable.Dispose();
				}
			}
		}
		finally
		{
			IDisposable disposable2 = image as IDisposable;
			if (disposable2 != null)
			{
				disposable2.Dispose();
			}
		}
	}
	[EntryPoint]
	public static int main(string[] argv)
	{
		Program.showForm(new Program.audioSample());
		return 0;
	}

Here you can clearly see how the types were written into the consuming application’s assembly and that each reference uses the provided types directly.

Advertisements

2 comments

Comments are closed.