Simon Fell > Its just code > .NET

I've seen a few questions on this recently (this is new for v14.0), so i thought I put together a sample of calling retrieve in the metadata file api to fetch a package of all your reports. This is in VS.NET 2005, but should be easy to translate to other environments. The code does the following steps (this is more complex that other cases because Reports don't support Wildcards)

1. Calls the partner api login to get a sessionId and the URL to the metadata API for your username.
2. Creates and configures an instance of the metadata client.
3. Calls ListMetadata to find out all the ReportFolders.
4. Using the list of ReportFolders, calls ListMetadata again to find out all the Reports in those ReportFolders.
5. Uses Retrieve / CheckStatus / CheckRetrieveStatus to fetch a package of the reports, and writes the package out to c:\reports.zip

using System;
using System.Collections.Generic;
using System.Text;

namespace mdRetrieve
{
    class Program
    {
        static void Main(string[] args)
        {
            // regular Enterprise/Partner Login call
	    // I added a WebReference of the partner wsdl as sf, and the Metadata WSDL as md
            sf.SforceService svc = new sf.SforceService();
            sf.LoginResult lr = svc.login(args[0], args[1]);

            // set up a MetdataService client
            md.MetadataService ms = new md.MetadataService();
            ms.SessionHeaderValue = new md.SessionHeader();
            ms.SessionHeaderValue.sessionId = lr.sessionId;
            ms.Url = lr.metadataServerUrl;

            Console.WriteLine("Logged in as {0}", lr.userInfo.userName);
            String [] reportFiles = ListReports(ms);
            RetrieveReports(ms, reportFiles);
        }


        static String [] ListReports(md.MetadataService ms)
        {
            // can't use wildcards with reports, so need to fetch the list
            // of ReportFolders first, then fetch all the reports in
            // each folder.
            md.ListMetadataQuery q = new md.ListMetadataQuery();
            q.type = "ReportFolder";
            md.FileProperties[] fp = ms.listMetadata(new md.ListMetadataQuery[] { q });
            if (fp == null)
            {
                Console.WriteLine("No report folders returned");
                return new String[0];
            }
            List reportFiles = new List();
            q.type = "Report";
            foreach (md.FileProperties p in fp)
            {
                q.folder = p.fullName;
                // listMetadata can take more than one item at a time
                // left as an exercise for the reader to batch up these calls.
                md.FileProperties[] rps = ms.listMetadata(new md.ListMetadataQuery[] { q });
                if (fp == null) continue;
                foreach (md.FileProperties rp in rps)
                {
                    Console.WriteLine("{0}", rp.fileName);
                    reportFiles.Add(rp.fullName);
                }
            }
            return reportFiles.ToArray();
        }

        static void RetrieveReports(md.MetadataService ms, String [] reportFiles) {
            // build up an unpackaged retrieve request for the list of reports.
            md.RetrieveRequest r = new md.RetrieveRequest();
            r.apiVersion = 14.0;
            r.unpackaged = new md.Package();
            md.PackageTypeMembers m = new md.PackageTypeMembers();
            m.name = "Report";
            m.members = reportFiles;
            r.unpackaged.types = new md.PackageTypeMembers[] { m };

            // start the retrieve request
            md.AsyncResult ar = ms.retrieve(r);
            // wait for it to complete, sleeping as necassary.
            while (!ar.done)
            {
                System.Threading.Thread.Sleep(ar.secondsToWait * 1000);
                ar = ms.checkStatus(new String[] { ar.id })[0];
            }

            // did it work ?
            if (ar.state == md.AsyncRequestState.Error)
                Console.WriteLine("{0} {1}", ar.statusCode, ar.message);
            else
            {
                // now actually go get the results 
                md.RetrieveResult rr = ms.checkRetrieveStatus(ar.id);
                if (rr.messages != null)
                    foreach (md.RetrieveMessage rm in rr.messages)
                        Console.WriteLine("{0} : {1}", rm.fileName, rm.problem);

                // write the zipFile out to a disk file.
                using (System.IO.FileStream fs = new System.IO.FileStream("c:\\reports.zip", System.IO.FileMode.Create))
                    fs.Write(rr.zipFile, 0, rr.zipFile.Length);
            }
        }
    }
}

I discovered a new Salesforce related blog today, and noticed that they spread the seeming popular meme that the metadata API doesn't work on standard objects, so just clarify that that's wrong, here's a .NET example that adds a new custom field to the standard Account object.

using System;
using System.Collections.Generic;
using System.Text;

namespace metadata
{
    class Program
    {
        static void Main(string[] args)
        {
            if (args.Length != 2)
            {
                Console.WriteLine("useage: metadataDemo username password");
                return;
            }
            MetadataCreator mc = new MetadataCreator(args[0], args[1]);
            mc.Create();
        }
    }

    class MetadataCreator
    {
        private metaforce.MetadataService ms;
        private sforce.SforceService ss;

        public MetadataCreator(String username, String password)
        {
            ss = new sforce.SforceService();
            sforce.LoginResult lr = ss.login(username, password);
            
            ss.Url = lr.serverUrl;
            ss.SessionHeaderValue = new sforce.SessionHeader();
            ss.SessionHeaderValue.sessionId = lr.sessionId;

            ms = new metaforce.MetadataService();
            ms.Url = lr.metadataServerUrl;
            ms.SessionHeaderValue = new metaforce.SessionHeader();
            ms.SessionHeaderValue.sessionId = lr.sessionId;
        }

        public void Create()
        {
            metaforce.CustomField cf = new metaforce.CustomField();
            cf.description = "Favorite fruit";
            cf.fullName = "Account.favFruit__c";
            cf.label = "Fav Fruit";
            cf.type = metaforce.FieldType.Text;
            cf.length = 25;
            cf.lengthSpecified = true;

            metaforce.AsyncResult r = ms.create(new metaforce.Metadata[] { cf })[0];
            while(!r.done) {
                System.Threading.Thread.Sleep(r.secondsToWait * 1000);
                r = ms.checkStatus(new string[] { r.id })[0];
            }
            if (r.state == metaforce.AsyncRequestState.Error) 
                Console.WriteLine("Error : {0} {1}", r.statusCode, r.message);
            else {
                Console.WriteLine("Done, added new field to Account");
            }
        }
    }
}

A few people have been asking for a complete example of using the metadata API from .NET, If you've using the regular enterprise/partner API, then the patterns should look familiar. One wrinkle for .NET is the whole fooSpecified mess, this is more apparent in the metadata API because most of the fields aren't strings.

I created a new project in VS.NET 2005, imported the enterprise WSDL (as sforce) and imported the metadata WSDL (as metaforce), the code makes a login call with the enterprise API, creates a stub for the metadata API, configuring the sessionHeader and Url from the login result (just like for the regular enterprise stub). then goes on to setup a CustomObject structure and passes it to create. One difference to the enterprise API is that the process is async, so you get back an AsyncResult structure that indicates its current state, a guess at how long to wait before checking its state again and so on, so we sit in a loop (with the specified waits) until its done. If the object was created, then we call describeSObject on the enterprise API to check that it really did do something, here's the code.

using System;
using System.Collections.Generic;
using System.Text;

namespace metadataDemo
{
    class Program
    {
        static void Main(string[] args)
        {
            if (args.Length != 2)
            {
                Console.WriteLine("useage: metadataDemo username password");
                return;
            }
            MetadataCreator mc = new MetadataCreator(args[0], args[1]);
            mc.Create();
        }
    }

    class MetadataCreator
    {
        private metaforce.MetadataService ms;
        private sforce.SforceService ss;

        public MetadataCreator(String username, String password)
        {
            ss = new sforce.SforceService();
            sforce.LoginResult lr = ss.login(username, password);
            
            ss.Url = lr.serverUrl;
            ss.SessionHeaderValue = new sforce.SessionHeader();
            ss.SessionHeaderValue.sessionId = lr.sessionId;

            ms = new metaforce.MetadataService();
            ms.Url = lr.metadataServerUrl;
            ms.SessionHeaderValue = new metaforce.SessionHeader();
            ms.SessionHeaderValue.sessionId = lr.sessionId;
        }

        public void Create()
        {
            metaforce.CustomObject co = new metaforce.CustomObject();
            co.deploymentStatus = metaforce.DeploymentStatus.Deployed;
            co.deploymentStatusSpecified = true;
            co.description = "My Custom Object created from .NET";
            co.fullName = "DotNetCustomObject__c";
            co.label = "DotNet Custom Object";
            co.pluralLabel = "DotNet Custom Objects";
            co.sharingModel = metaforce.SharingModel.ReadWrite;
            co.sharingModelSpecified = true;
            co.nameField = new metaforce.CustomField();
            co.nameField.type = metaforce.FieldType.Text;
            co.nameField.label = "The Name";
            co.nameField.length = 100;
            co.nameField.lengthSpecified = true;

            metaforce.AsyncResult r = ms.create(new metaforce.Metadata[] { co })[0];
            while(!r.done) {
                System.Threading.Thread.Sleep(r.secondsToWait * 1000);
                r = ms.checkStatus(new string[] { r.id })[0];
            }
            if (r.state == metaforce.AsyncRequestState.Error) 
                Console.WriteLine("Error : {0} {1}", r.statusCode, r.message);
            else {
                Console.WriteLine("Created customObject {0}, calling describeSObject now", co.fullName);
                sforce.DescribeSObjectResult d = ss.describeSObject(co.fullName);
                Console.WriteLine("DescribeSobject for {0} ({1})", d.name, d.label);
                foreach(sforce.Field f in d.fields)
                    Console.WriteLine("{0}\t{1}", f.name, f.type);
            }
        }
    }
}

And here's the output from a running it

Created customObject DotNetCustomObject__c, calling describeSObject now
DescribeSobject for DotNetCustomObject__c (DotNet Custom Object)
Id      id
OwnerId reference
IsDeleted       boolean
Name    string
CreatedDate     datetime
CreatedById     reference
LastModifiedDate        datetime
LastModifiedById        reference
SystemModstamp  datetime

Creating customFields is just as easy, just remember to specify the fully qualified fieldname as its fullName, e.g. if you wanted a new field on your custom object co__c, you set the name to be co__c.newField__c, or if you wanted a new custom field on Account, it'd be Account.newField__c. Share and Enjoy.

Does Windows Home Server really work with Time Machine ? Paul Thurrott's WHS review hints that it does, "Mac user? You can access the WHS shared folders as you would any other Windows share, and that means your backup program--like Apple's Backup--can use a share as a save location as well. "We're a great back-end store for Time Machine," Headrick told me, alluding to the new backup feature Apple recently announced for Mac OS X Leopard." But, the Apple Time Machine page says you need a HFS+ disk, or another machine running Leopard, and Microsoft's own WHS page says "Mac users have the ability to save to and remotely access Windows Home Server." which i take to mean you can use the existing Windows/OSX Samba support.

This (mindnumbly stupid move IMO, YMMV) doesn't seem very conducive to making this a success, perhaps everyone is hoping they'll of forgotten by the time it actually ships.

Downloading Visual Studio 2005 Service Pack 1 (update 4 of 6)...
27.50MB of 432.00 MB



Holly cow, is it a service pack, or a whole new dev environment???

Scott Hanselman is again doing the Diabetes Walk, and this year is stepping up his goal to $50,000, go ahead and help him meet the goal. To the Salesforce.com folks out there, remember that they'll match your contribution, and I'm sure many other employers do as well, an easy way to make your money go twice as far.

As I mentioned way back in April 2003, wsdl.exe /server is basically useless. This is fixed in .NET 2.0, and wsdl.exe /serverInterface works pretty much as you'd expect (or at as I'd expect, which is good enough for me). If you're trying to use the new Outbound Messaging feature that was part of the recent Winter release, and you're still on .NET 1.1, then you'll run into this real fast. For those who haven't seen the feature, basically you specify some condition to send you a message, you define what fields you want in the message from the target object, and we generate a WSDL that represents the messages we'll send you. You feed this wsdl into your tool of choice to spit out your server skeleton, and off you go. Well, unless you're on .NET 1.1, where its not remotely obvious what to do. So, here's a quick guide to getting up and running with Outbound Messaging and .NET 1.1.

• Stop, why are you still on .NET 1.1? are you sure you can't move up to 2.0, you should seriously investigate this first.
• Ok, if you're still here, go into the Salesforce.com app, setup, workflow and setup your rule and outbound message, in the message pick the fields you want (I picked the Contact object and the firstName & lastName fields).
• Right click on the "Click for WSDL" link, and save it somewhere.
• From the command line, run wsdl.exe /server om.wsdl to generate the stub class and types file (it'll be called NotificationService.cs by default)
• You'll notice that the generated class is abstract, but don't fall for the trick of creating a concrete subclass, its not going to work.
• Horrible as it is, the easiest way forward is to now modify the generated code, remove the abstracts, and add an implementation to the notifications method, here i'll just copy the contact objects to a static list so i can have another page view the list of contacts i've received.

  [System.Web.Services.WebServiceBindingAttribute(Name="NotificationBinding", Namespace="http://soap.sforce.com/2005/09/outbound")]
  [System.Xml.Serialization.XmlIncludeAttribute(typeof(sObject))]
  public class NotificationService : System.Web.Services.WebService {
      
      /// 
      [System.Web.Services.WebMethodAttribute()]
      [System.Web.Services.Protocols.SoapDocumentMethodAttribute("", 
  		Use=System.Web.Services.Description.SoapBindingUse.Literal, 
  		ParameterStyle=System.Web.Services.Protocols.SoapParameterStyle.Bare)]
      [return: System.Xml.Serialization.XmlElementAttribute("notificationsResponse", Namespace="http://soap.sforce.com/2005/09/outbound")]
      public notificationsResponse notifications(
  		[System.Xml.Serialization.XmlElementAttribute("notifications", Namespace="http://soap.sforce.com/2005/09/outbound")] notifications n) {
      	foreach(ContactNotification cn in n.Notification) {
      		lock(objects) {
      			objects.Add(cn.sObject);
      		} 
  	}
  	notificationsResponse r = new notificationsResponse();
  	r.Ack = true;
  	return r;
      }
  
      private static System.Collections.ArrayList objects = new System.Collections.ArrayList();
      
      public static System.Collections.ArrayList CurrentObjects() {
      	lock(objects) {
      		return (System.Collections.ArrayList)objects.Clone();
      	}
      }
  }

• Next up you need to compile this into a dll, run csc /t:library /out:bin/om.dll NotificationService.cs (this assumes you've already created the bin directory that asp.net needs). Now you can wire it up as a webservice, you can do this by editing the web.config to map a URI to the class, or just use a dummy asmx page that points the class, that's what i did, here's om.asmx

  <%@ WebService class="NotificationService" %>

• At this point you should be able hit hit the asmx page with a browser and get the regular .NET asmx service page.
• Next, we'll add a simple aspx page that can access the static objects collection and print out the details from the messages it got. (obviously a real implementation should do some real work in the notifications method, but this is a handy test)

  <%@ Page language="C#" %>
  <html><head>
  </head><body>
  <h1>Recieved Contact ids</h1>
  <%
  foreach(Contact c in NotificationService.CurrentObjects()) {
  	Response.Write(string.Format("{0} {1} {2}<br>", c.Id, Server.HtmlEncode(c.FirstName), Server.HtmlEncode(c.LastName)));
  }
  %>
  </body></html>

• That's it, go into the app, trigger your message, and refresh the aspx page to see the data from the message we sent you.

A slightly cleaner approach is to actually do the subclass, and to copy over all the class and method attributes to the subclass as well, at least then you can easily re-run wsdl.exe if you need to (say if you changed the set of selected fields).

Over on ADN there's a tech note that explains how to add request & response compression support to .NET generated web services clients. As .NET 2.0 provides support for response compression out the box now (once you've turned it on), you'll find this code fails on .NET 2.0 because it ends up trying to decompress the response twice. So I updated the code for .NET 2.0, it now only has to handle compressing the request. The nice thing about the approach it takes (subclassing the generated proxy class), is that you don't have to change the generated code at all, so it doesn't matter how many times you do update web reference, you'll still be in compressed goodness. So, all you need is this one subclass wrapper around your generated proxy class and you're good to go.

    class SforceGzip : sforce.SforceService
    {
        public SforceGzip()
        {
            this.EnableDecompression = true;
        }

        protected override System.Net.WebRequest GetWebRequest(Uri uri)
        {
            return new GzipWebRequest(base.GetWebRequest(uri));
        }
    }

Then in your code that uses the proxy, just create an instance of this puppy instead of the regular proxy, e.g.

    sforce.SforceService svc = new SforceGzip();
    sforce.LoginResult lr = svc.login(args[0], args[1]);

All this is included in the sample project in the download, share and enjoy, share and enjoy

In his post Trade-Offs, Sam points to my post last year on trying to have SOAP headers with simple types in .NET. In the comments Asbjørn Ulsberg claims that the behavior is related to handling nulls, and that its fixed in .NET 2.0. Well, lets see shall we.

To start with, the header defined is not nillable, therefore that's a complete read herring. as a reminder, here's the element definition that's detailed in the post and in the WSDL.

<s:element name="ValidFrom" type="s:dateTime"/>

I fired up .NET 2.0's WSDL.exe tool and pointed it at the first wsdl (you can play along at home if you want to)

C:\sampleCode\dotnet20_headers>wsdl http://www.pocketsoap.com/weblog/2005/08/test_1header.wsdl
Microsoft (R) Web Services Description Language Utility
[Microsoft (R) .NET Framework, Version 2.0.50727.42]
Copyright (C) Microsoft Corporation. All rights reserved.
Writing file 'C:\sampleCode\dotnet20_headers\TestService.cs'.

And lets take a look at the generated code

[System.CodeDom.Compiler.GeneratedCodeAttribute("wsdl", "2.0.50727.42")]
[System.SerializableAttribute()]
[System.Diagnostics.DebuggerStepThroughAttribute()]
[System.ComponentModel.DesignerCategoryAttribute("code")]
[System.Xml.Serialization.XmlTypeAttribute(Namespace="http://www.w3.org/2001/XMLSchema")]
[System.Xml.Serialization.XmlRootAttribute("ValidFrom", Namespace="http://test.sforce.com/", IsNullable=false)]
public partial class dateTime : System.Web.Services.Protocols.SoapHeader {
    
    private string[] textField;
    
    /// 
    [System.Xml.Serialization.XmlTextAttribute()]
    public string[] Text {
        get {
            return this.textField;
        }
        set {
            this.textField = value;
        }
    }
}

Looks remarkably like the .NET 1.1 version (except for the fields vs properties thing), so, no, not fixed, not any better.

Apparently the problem with .NET remoting putting char 0 in a fault message from 4.5 years ago is still an issue. (and no, there's still no magic switch to allow PocketSOAP to parse stuff that looks like it might be XML, but isn't XML).

The MSDN library is littered with sample code, most of it annoys the hell out me because it doesn't say what the expected outcome is from running the code, and sometimes the samples are downright misleading, take this one from CodeTypeReferenceCollection

// Creates an empty CodeTypeReferenceCollection.
CodeTypeReferenceCollection collection = new CodeTypeReferenceCollection();

// Adds a collection of CodeTypeReference objects to the collection.
CodeTypeReferenceCollection referencesCollection = new CodeTypeReferenceCollection();
referencesCollection.Add( new CodeTypeReference(typeof(bool)) );
referencesCollection.Add( new CodeTypeReference(typeof(bool)) );
collection.AddRange( referencesCollection );

// Tests for the presence of a CodeTypeReference in the 
// collection, and retrieves its index if it is found.
CodeTypeReference testReference = new CodeTypeReference(typeof(bool));
int itemIndex = -1;
if( collection.Contains( testReference ) )
    itemIndex = collection.IndexOf( testReference );

No where does it mention what value itemIndex might have when you run this. I saw this and took it to mean that the CodeTypeReference's would get compared by value rather than reference, and so find the bool CodeTypeReference even though its a different instance, but no, running the code reveals that CodeTypeReference doesn't seem to override Equals or ==, and so the collection.Contains call returns false. grrrhhh. (this makes finding out if a particular type is in the Collection something of a PIA if you didn't construct and populate the collection yourself.)

Getting no where with my woes with SchemaImporterExtension, despite the claims in the docs, it seems like its only really designed to substitute out types for existing types that are referenced in, not for more complex code generation solutions. There's a bunch of trivial examples out there, but nothing that does anything complex with it. Looks like a more workable approach will be to build a replacement wsdl.exe (which is largely a wrapper around ServiceDescriptionImporter. Its a pity that everything except Main is private in the wsdl.exe class, it always seems like the Microsoft classes always default to not being re-usable unless they really want it to, which sucks, i get to duplicate a whole bunch of boring boiler plate code just so I can get at the CodeDom before it gets serialized out to a file.

Update So I still need to wire up all the wsdl.exe options, but I got the basic's working, it sucks in the WSDL, massages my changes into the CodeDom then spits it out to a file :)

I've been digging into SchemaImporterExtension to see if I can fix the generated code from .NET's wsdl.exe to include setting the FooSpecified flag from the foo Setter (see this post from way back in 2004 if you need a refresher on the details). What's currently generated for an optional element (minOccurs='0' / maxOccurs='1') is this

private System.Nullable<System.DateTime> createdDateField;
    private bool createdDateFieldSpecified;

    [System.Xml.Serialization.XmlElementAttribute(IsNullable=true)]
    public System.Nullable<System.DateTime> CreatedDate {
        get {
            return this.createdDateField;
        }
        set {
            this.createdDateField = value;
        }
    }

And what It really should of been is

private System.Nullable<System.DateTime> createdDateField;
    private bool createdDateFieldSpecified;

    [System.Xml.Serialization.XmlElementAttribute(IsNullable=true)]
    public System.Nullable<System.DateTime> CreatedDate {
        get {
            return this.createdDateField;
        }
        set {
            this.createdDateField = value;
            this.createdDateFieldSpecified = true;
        }
    }

Its frustrating, I'm sure someone up in Redmond could fix this in a couple of hours, but I've been waiting for 2 years now (and I expect the wait will be a whole bunch longer). But with .NET 2.0, SchemaImporterExtension gives hope that I can fix this myself without having to wait. Unfortuantly the samples and docs are pretty skinny, it seems like you get a chance to replace the code generation for a particular type, but you don't seem to get any access to the default code that would be generated, this means in this case you need to duplicate all of the logic that generates the code for a ComplexType. As anyone who's done work with generating code from WSDL will tell you, this is a non trivial amount of work to get right. Anway, I plowed in and got the basics working, however have run into a couple of road blocks, how do I tell the importer to import an additional type (e.g. complexType foo has an element of type bar, so I need to import the bar type as well), and have it generate its code / call back into the extensions etc ? You get passed an instance of a XmlSchemaImporter object to your code, (but this is doc'd as "This class supports the .NET Framework infrastructure and is not intended to be used directly from your code." huh? so why is it in the methods in a public extension point?) anyway, this seems to have the hopeful method called ImportSchemaType (amongst others), however all this seems to do is generate the naming mappings between the schema type QName you pass it, and the resulting CLR type, it doesn't generate any code in the final proxy. I'm also seeing that ImportSchemaType gets called for some schemaTypes (in particular complexTypes that are used by other complexTypes), and the resulting additions to the CodeNamespace seem to get dropped.).

I put together a quick sample that re-pro's the latter problem with code getting dropped, here's the ImporterExtension, a test WSDL, parameters file for wsdl.exe and the resulting generated file, notice that even though ImportSchemaType was called for the Bar type, and we added a CodeTypeDeclaration to the mainNamespace for it, it doesn't appear in the final code.

C:\Source\dotnet\sfSchemaImporterTest>wsdl.exe /par:params.xml
Microsoft (R) Web Services Description Language Utility
[Microsoft (R) .NET Framework, Version 2.0.50727.42]
Copyright (C) Microsoft Corporation. All rights reserved.
Generating type Bar
adding Bar to mainNamespace
Generating type FooResult
adding FooResult to mainNamespace
Writing file 'C:\Source\dotnet\sfSchemaImporterTest\DoFooService.cs'.

This also only seems to allow me the chance to fiddle with the schema types, is there an equvilent extension to allow me to fiddle with the web services specifics bits? (like the DoFooService class in the above sample, or the classes generated for soap headers)