Sunday, December 8, 2013

Simple dependency management for dependent Salesforce objects

Introduction

Salesforce programmers know it is sometime difficult to save multiple objects with dependencies on each other in the right order and with as little effort possible.  The "Collecting Parameter" pattern is an easy way to do this and this article will show you how to use it in your own code.

Unit of Work

In June 2013, FinancialForce's CTO, Andrew Facett, wrote his Unit Of Work article, explaining how a dependency mechanism might be implemented to simplify the saving of multiple objects with dependencies between them.

The problem is a common one for Salesforce programmers--the need to create master and detail objects simultaneously.  Programmers must save the master objects first before their IDs can be set in the detail objects.

An example might be an invoice and its line-items.  To save any InvoiceLine__c, its master object, an Invoice__c, must be saved first.

To solve this problem, Xede uses a pattern popularized by Kent Beck in his 1995 book, Smalltalk Best Practice Patterns, called Collecting Parameter.  For those unfamiliar with the Smalltalk programming language, it can be briefly described as the first object-oriented language where everything is an object.  Numbers, messages, classes, stack frames--everything.  In 1980, (nearly 34 years ago) it also supported continuable exceptions and lambda expressions.    Lest I gush too much about it, I'll say only that nearly all object oriented languages owe their best features to Smalltalk and their worse features to either trying to improve on it or ignoring prior art.

Returning to the subject at-hand, dependency saves, Xede will have created two classes to wrap the objects; Invoice and InvoiceLine.  Each instance of Invoice will aggregate within it the InvoiceLine instances belonging to it.

The code might look something like this.

// create an invoice and add some lines to it
Invoice anInvoice = new Invoice(anInvoiceNumber, anInvoiceDate, aCustomer);
...
// adding details is relatively simple
anInvoice.add(anInvoiceLine);
anInvoice.add(anotherInvoiceLine);
anInvoice.save();

So now our Invoice has two detail instances inside it.  Keeping true to the OO principles of data-hiding and loose-coupling, we can safely ignore how these instances store their sobject variables;  Invoice's Invoice__c and InvoiceLine's InvoiceLine__c.  But without knowing how they store their sobjects, how can we save the master and the detail records with the minimum of two DMLs, one to save the master and another to save the details?

We do it using a collecting parameter.

Collecting Parameter

A collecting parameter is basically a collection of like-things that cooperating classes add to.  Imagine a basket that might get passed to attendees at a charity event.  Each person receiving the collection basket may or may not add cash or checks to it.  In both programming and charity fundraisers it is better manners to let each person add to the basket themselves than to have an usher reach into strangers' pockets and remove cash.  The latter should be regarded as criminal--if not at charity events then in programming.

For programmer's such a thing violates data-hiding; not all classes keep their wallets in the same pocket (variable), some may use money clips rather than wallets, some use purses (collection types), some may have cash while others have checks or coin.  Writing code that will rummage through each class' data looking for cash is nearly impossible--even with reflection.  In the end they all get deposited into a bank account.

Let's first look at the saveTo() methods of Invoice and InvoiceLine.  They are the simplest.

public with sharing class Invoice {
    public getId() { return sobjectData.id; }

    public override void saveTo(list<sobject> aList, list<XedeObject> dependentList)
    {
        aList.add(sobjectData);

        for (InvoiceLine each : lines)
            saveTo(aList, dependentList);
    }

    Invoice__c sobjectData;
    list<InvoiceLine> lines;
}

Invoice knows where it keeps its own reference to Invoice__c (cohesion), so when it comes time to save it simply adds its sobject to the list of sobjects to be saved.  After that, it also knows where it keeps its own list of invoice lines and so calls saveTo() on each of them.

public with sharing class InvoiceLine {
    public override void saveTo(list dependentList) {
        if (sobjectData.parent__c != null)  // if I already have my parent's id I can be saved
            aList.add(sobjectData);

        else if (parent.getId() != null) {  // else if my parent has an id, copy it and I can be saved
            sobjectData.parent__c = parent.getId();
            aList.add(sobjectData);
        }

        else
            dependentList.add(this); // I can't be saved until my parent is saved
    }

    Invoice parent;
    InvoiceLine__c sobjectData;
}

InvoiceLine's implementation is nearly as simple as Invoice's, but subtly different.

Basically, if the InvoiceLine already has its parent's id or can get it's parent's id, then it adds its sobject data to the list to be saved.  If it doesn't have its parent's id then it must wait its turn, and adds itself to the dependent list.

Reader's may wonder why Invoice doesn't decide for itself whether to save its children.  Invoice could skip sending saveTo() to its children if it doesn't have an id, but whether or not its children should be saved is not its decision--it's theirs.  They may have other criteria that must be met before they can be saved.  They may have two master relationships and are waiting for them both.  They may have rows to delete before they can be saved, or may have detail records of their own with other critiera independent of whether Invoice has an id or not.  Whatever the reason may be, the rule is each objects should decide for itself whether it's ready to save, just as it's each person's decisions whether and how much money to put into the collection basket.

In our example below, save() passes two collection baskets; one collects sobjects and another collections instances of classes whose sobjects aren't ready for saving--yet.  save() loops over both lists until they're empty, and in this way is able to handle arbitrary levels of dependencies with the minimum number of DML statements.

Let's look at the base class' (XedeObject) implementation of save().

public virtual class XedeObject {
    public virtual void save() {
        list objectList = new list();
        list dependentList = new list { this };

        do {
            List aList = new List();
            List updateList = new List();
            List insertList = new List();

            objectList = new list(dependentList);
            dependentList.clear();

            for (XedeObject each: objectList)
                each.saveTo(aList, dependentList);

            for (sobject each : aList) {
                if (each.id == null)
                    insertList.add(each);
                else
                    updateList.add(each);
            }

            try {
                update updateList;
                insert insertList;
            } catch (DMLException dmlex)  {
                XedeException.Raise('Error adding or updating object : {0}', dmlex.getMessage());
            }
        } while (dependentList.isEmpty() == false);
    }

    public virtual void saveTo(list anSobjectList, list aDependentList)
    {
        subclassMethodError();
    }
}

To understand how this code works you need to be familiar with subclassing.  Essentially, the classes Invoice and InvoiceLine are both subclasses of XedeObject.  This means they inherit all the functionality of XedeObject.  Though neither Invoice or InvoiceLine implement save(), they will both understand the message because they've inherited its implementation from XedeObject.

The best way to understand what save() does is to walk through "anInvoice.save()."

anInvoice.save() executes XedeObject's save() method because Invoice doesn't have one of its own (remember it's a subclass of XedeObject).  save() begins by adding its own instance to dependentList.  Then it loops over the dependent list, sending saveTo() to each instance, and collecting new dependent objects in the dependent list.

After collecting all the objects it either updates or saves them, then returns to the top of the list of the dependent list isn't empty and restarts the process.

When the dependent list is empty there's nothing else to do and the methods falls off the bottom returning to the caller.

XedeObject also implements saveTo(), but its implementation throws an exception.  XedeObject's subclasses ought to implement saveTo() themselves if they intend to participate in the dependency saves.  If they don't or won't, there's no need to override saveTo().

One of our recent projects was a loan servicing system.  Each loan could have multiple obligors, and each obligor could have multiple addresses.  The system could be given multiple loans at a time to create, and with each batch of loans a log record was recorded.  We had an apiResponse object with a list of loans.  When we called anApiResponse.save(), it's saveTo() sent saveTo() to each of its loans, each loan sent saveTo() to each of its obligors, and each obligor() sent saveTo() to each of its addresses, before apiResponse sent saveTo() to its log class.

In the end, ApiResponse saved the loans, obligors, addresses, and log records with three DML statements--all without anything much more complicated than each class implementing saveTo().

Some programmers may argue that interfaces might have accomplished the same feat without subclassing, but in this case it is not true.  Interfaces don't provide method implementation.  Had we used interfaces then every object would be required to implement save().

Still to do

As useful as save()/saveTo() has proved to be, I can think of a few improvements I'd like to make to it.

First, I'd like to add a delete list.  Some of our operations include deletes, and rather than having each object do its own deletes I'd prefer to collect them into a single list and delete them all at once.

Next, the exception handling around the update and insert should be improved.  DmlException has lots of valuable information we could log or include in our own exception.

Third, I would love to map the DML exceptions with the objects that added them to the list.  save() could then collect all the DML exceptions and send them to the objects responsible for adding them to the list.

Coming up

  • XedeObject has implemented other useful methods we tend to use in multiple of our projects.  Implementing them once in XedeObject and deploying it to each of our customer's orgs saves time, money, and improves consistency across all our projects.  One of these is coalesce().  There are many others.
  • Curl scripts for exercising Salesforce REST services.
  • Using staticresources as a source for unit-test data.

Friday, March 22, 2013

A better way to generate XML on Salesforce using VisualForce

There are easier ways to generate XML on Salesforce than either the Dom library or XmlStreamWriter class.  If you've done either, perhaps you'll recognize the code below.

public static void DomExample()
{
    Dom.Document doc = new Dom.Document();
    
    Dom.Xmlnode rootNode = doc.createRootElement('response', null, null);

    list accountList = [ 
        select  id, name, 
                (select id, name, email from Contacts) 
          from  Account 
    ];
          
    for (Account eachAccount : accountList) {
        Dom.Xmlnode accountNode = rootNode.addChildElement('Account', null, null);
        accountNode.setAttribute('id', eachAccount.Id);
        accountNode.setAttribute('name', eachAccount.Name);
        
        for (Contact eachContact : eachAccount.Contacts) {
            Dom.Xmlnode contactNode = accountNode.addChildElement('Contact', null, null);
            contactNode.setAttribute('id', eachContact.Id);ac
            contactNode.setAttribute('name', eachContact.Name);
            contactNode.setAttribute('email', eachContact.Email);
        }
    }
    
    system.debug(doc.toXmlString());            
}

Or maybe this example.

public static void StreamExample()
{
    XmlStreamWriter writer = new XmlStreamWriter();
    
    writer.writeStartDocument('utf-8', '1.0');        
    writer.writeStartElement(null, 'response', null);
    
    list accountList = [ 
        select  id, name, 

                (select id, name, email from Contacts) 
          from  Account 
    ];
          
    for (Account eachAccount : accountList) {
        writer.writeStartElement(null, 'Account', null);
        writer.writeAttribute(null, null, 'id', eachAccount.Id);
        writer.writeAttribute(null, null, 'name', eachAccount.Name);        

        for (Contact eachContact : eachAccount.Contacts) {
            writer.writeStartElement(null, 'Contact', null);
            
            writer.writeAttribute(null, null, 'id', eachContact.Id);
            writer.writeAttribute(null, null, 'name', eachContact.Name);
            writer.writeAttribute(null, null, 'email', eachContact.Email);
            
            writer.writeEndElement();
        }
        
        writer.writeEndElement();
    }
    
    writer.writeEndElement();
    
    system.debug(writer.getXmlString());
    
    writer.close();            
}

But wouldn't you rather write something like this?

public static void PageExample()
{
    PageReference aPage = Page.AccountContactsXML;
    aPage.setRedirect(true);
    system.debug(aPage.getContent().toString());
}

Let's take a look at what makes creating the XML possible with so few lines of Apex.

Rather than build our XML using Apex code, we can type it directly into a Visualforce page--providing we strip all VF's page accessories off using apex:page attributes.


<apex:page StandardController="Account" recordSetVar="Accounts" contentType="text/xml" showHeader="false" sidebar="false" cache="false">
<?xml version="1.0" encoding="UTF-8" ?>
<response>
<apex:repeat value="{!Accounts}" var="eachAccount" >
    <Account id="{!eachAccount.id}" name="{!eachAccount.name}">&
    <apex:repeat value="{!eachAccount.contacts}" var="eachContact">
        <Contact id="{!eachContact.id}" name="{!eachContact.name}" email="{!eachContact.email}"/>
    </apex:repeat>
    </Account>
</apex:repeat>
</response>
</apex:page>

The secret that makes this code work is setting the page's API version 19.0 inside its metadata.  That is the only thing that allows the <?xml ?> processing instruction to appear at the top without the Visualforce compiler throwing Conniptions (a subclass of Exception). 

Depending on how much XML you need to generate, another advantage to the VisualForce version is how few script statements are required to produce it.

Number of code statements: 4 out of 200000

Our Dom and Stream examples require 28 and 37 respectively--and that's in a developer org with only three accounts and three contacts.  Additionally, the Page example is only 18 lines including both the .page and .cls, whereas the Dom and Stream examples are 27 and 38 lines respectively (coincidence?).

But what happens when we add billing and shipping addresses (and two more contacts)?

Our page example's Apex code doesn't change, but its page does.

<apex:page StandardController="Account" recordSetVar="Accounts" contentType="text/xml" showHeader="false" sidebar="false" cache="false">
<?xml version="1.0" encoding="UTF-8" ?>
<response>
<apex:repeat value="{!Accounts}" var="eachAccount" >    
    <Account id="{!eachAccount.id}" name="{!eachAccount.name}">
        <apex:outputPanel rendered="{!!IsBlank(eachAccount.billingStreet)}" layout="none">
            <Address type="Billing">
                <Street>{!eachAccount.billingStreet}</Street>
                <City>{!eachAccount.billingCity}</City>
                <State>{!eachAccount.billingState}</State>
                <PostalCode>{!eachAccount.billingPostalCode}</PostalCode>
                <Country>{!eachAccount.billingCountry}</Country>
            </Address>        
        </apex:outputPanel>        
        <apex:outputPanel rendered="{!!IsBlank(eachAccount.shippingStreet)}" layout="none">            
            <Address type="Shipping">
                <Street>{!eachAccount.shippingStreet}</Street>
                <City>{!eachAccount.shippingCity}</City>
                <State>{!eachAccount.shippingState}</State>
                <PostalCode>{!eachAccount.shippingPostalCode}</PostalCode>
                <Country>{!eachAccount.shippingCountry}</Country>
            </Address>
        </apex:outputPanel>
        <apex:repeat value="{!eachAccount.contacts}" var="eachContact">&
            <Contact id="{!eachContact.id}" name="{!eachContact.name}" email="{!eachContact.email}"/>
        </apex:repeat>
    </Account>
</apex:repeat>
</response>
</apex:page>

We've added sections for both the billing and shipping codes, with conditional rendering in-case either doesn't exist.  In addition to our six lines of Apex (PageExample() above) we've added 12 new lines to the earlier 18 for a total of 36 lines.  The best part is, even with the extra XML being generated our Page example will still only consume 4 script statements of the already-insufficient 200,000.

How do our Dom and Stream examples fair?  Both are pasted together below into a single code section.

public static void DomExample()
{
    Dom.Document doc = new Dom.Document();        
    
    Dom.Xmlnode rootNode = doc.createRootElement('response', null, null);

    list accountList = [ 
        select    id, name, 
                billingStreet, billingCity,
                billingState, billingPostalCode,
                billingCountry,
                shippingStreet, shippingCity,
                shippingState, shippingPostalCode,
                shippingCountry,
                (select id, name, email from Contacts) 
          from    Account ];
          
    for (Account eachAccount : accountList) {
        Dom.Xmlnode accountNode = rootNode.addChildElement('Account', null, null);
        accountNode.setAttribute('id', eachAccount.Id);
        accountNode.setAttribute('name', eachAccount.Name);
        
        if (String.IsNotBlank(eachAccount.billingStreet)) {
            Dom.Xmlnode addressNode = accountNode.addChildElement('Address', null, null);
            addressNode.setAttribute('type', 'Billing');
            addressNode.addChildElement('Street', null, null).addTextNode(eachAccount.billingStreet);
            addressNode.addChildElement('City', null, null).addTextNode(eachAccount.billingCity);
            addressNode.addChildElement('State', null, null).addTextNode(eachAccount.billingState);
            addressNode.addChildElement('PostalCode', null, null).addTextNode(eachAccount.billingPostalCode);
            addressNode.addChildElement('Country', null, null).addTextNode(eachAccount.billingCountry);                
        }
        
        if (String.IsNotBlank(eachAccount.ShippingStreet)) {                
            Dom.Xmlnode addressNode = accountNode.addChildElement('Address', null, null);
            addressNode.setAttribute('type', 'Shipping');
            addressNode.addChildElement('Street', null, null).addTextNode(eachAccount.shippingStreet);
            addressNode.addChildElement('City', null, null).addTextNode(eachAccount.shippingCity);
            addressNode.addChildElement('State', null, null).addTextNode(eachAccount.shippingState);
            addressNode.addChildElement('PostalCode', null, null).addTextNode(eachAccount.shippingPostalCode);
            addressNode.addChildElement('Country', null, null).addTextNode(eachAccount.shippingCountry);                
        }
        
        for (Contact eachContact : eachAccount.Contacts) {
            Dom.Xmlnode contactNode = accountNode.addChildElement('Contact', null, null);
            contactNode.setAttribute('id', eachContact.Id);
            contactNode.setAttribute('name', eachContact.Name);
            contactNode.setAttribute('email', eachContact.Email);
        }
    }
    
    system.debug(doc.toXmlString());            
}

public static void StreamExample()
{
    XmlStreamWriter writer = new XmlStreamWriter();
    
    writer.writeStartDocument('utf-8', '1.0');        
    writer.writeStartElement(null, 'response', null);
    
    list accountList = [ 
        select    id, name, 
                billingStreet, billingCity,
                billingState, billingPostalCode,
                billingCountry,
                shippingStreet, shippingCity,
                shippingState, shippingPostalCode,
                shippingCountry,
                (select id, name, email from Contacts) 
          from    Account ];
          
    for (Account eachAccount : accountList) {
        writer.writeStartElement(null, 'Account', null);
        writer.writeAttribute(null, null, 'id', eachAccount.Id);
        writer.writeAttribute(null, null, 'name', eachAccount.Name);
        
        if (String.IsNotBlank(eachAccount.billingStreet)) {
            writer.writeStartElement(null, 'Address', null);
            writer.writeAttribute(null, null, 'type', 'Billing');                
            
            writer.writeStartElement(null, 'Street', null);
            writer.writeCharacters(eachAccount.billingStreet);
            writer.writeEndElement();
            
            writer.writeStartElement(null, 'City', null);
            writer.writeCharacters(eachAccount.billingCity);
            writer.writeEndElement();
            
            writer.writeStartElement(null, 'State', null);
            writer.writeCharacters(eachAccount.billingState);
            writer.writeEndElement();
            
            writer.writeStartElement(null, 'PostalCode', null);
            writer.writeCharacters(eachAccount.billingPostalCode);
            writer.writeEndElement();
            
            writer.writeStartElement(null, 'Country', null);
            writer.writeCharacters(eachAccount.billingCountry);
            writer.writeEndElement();

            writer.writeEndElement();                
        }
        
        if (String.IsNotBlank(eachAccount.shippingStreet)) {
            writer.writeStartElement(null, 'Address', null);
            writer.writeAttribute(null, null, 'type', 'Shipping');                
            
            writer.writeStartElement(null, 'Street', null);
            writer.writeCharacters(eachAccount.shippingStreet);
            writer.writeEndElement();
            
            writer.writeStartElement(null, 'City', null);
            writer.writeCharacters(eachAccount.shippingCity);
            writer.writeEndElement();
            
            writer.writeStartElement(null, 'State', null);
            writer.writeCharacters(eachAccount.shippingState);
            writer.writeEndElement();
            
            writer.writeStartElement(null, 'PostalCode', null);
            writer.writeCharacters(eachAccount.shippingPostalCode);
            writer.writeEndElement();
            
            writer.writeStartElement(null, 'Country', null);
            writer.writeCharacters(eachAccount.shippingCountry);
            writer.writeEndElement();

            writer.writeEndElement();                
        }

        for (Contact eachContact : eachAccount.Contacts) {
            writer.writeStartElement(null, 'Contact', null);
            
            writer.writeAttribute(null, null, 'id', eachContact.Id);
            writer.writeAttribute(null, null, 'name', eachContact.Name);
            writer.writeAttribute(null, null, 'email', eachContact.Email);
            
            writer.writeEndElement();
        }
        
        writer.writeEndElement();
    }
    
    writer.writeEndElement();
    
    system.debug(writer.getXmlString());
    
    writer.close();            
}

Our Dom example is 52 lines and takes 60 script statements and our Stream example has ballooned to 96 lines and takes 104 script statements on our tiny data set.  For anyone keeping track, PageExample() has 30% fewer lines than DomExample() and 63% fewer lines than StreamExample().  Most importantly, no matter how much data is involved, PageExample will only ever use 4 script statements while the other two will scale gometrically as each new row of data requires more than one script statement to generate.

Caveats and disclaimers

  • The page above is about as basic as I could come up.  It stands alone and requires no controllers.  Readers should be able to paste it directly into their development orgs and see what they get (dont' forget to set the API to 19.0).
  • So basic a page doesn't take into account ordering of the data.  If the XML data needs to be in a specific order a controller would be required to return that list back to the page using a SOQL "order by" clause.
  • Though this technique is great for generating XML it can't consume XML.  That's probably obvious to programmers but is important to point-out for management types that may visit.
  • XSL stylesheets can be easily reference from the XML page by simply adding <?xml-stylesheet type="text/xsl" href="..."?> after the <?xml ?> instruction.  Such a thing can be done with PageExample() and StreamExample(), but the Dom classes don't allow adding processing instructions that I know of.
  • It's impossible to use getContent() inside test methods. 

Note: This article was originally published March 22, 2013 at it.toolbox.com on Anything Worth Doing.

Follow @TomGagne